Job ID: NC-760037 (97590331)
Hybrid/Local Data Engineer (12+) with ETL, QA, Python, PyPI, SQL, DevOps and cybersecurity experience
Location: Raleigh NC(NCDPI)
Duration: 12 Months
***The manager would like the candidate to be onsite regularly throughout the position. The amount of onsite time per week will be negotiable. However, weekly onsite time will be required.
Skills:
Demonstrated mid-level+ experience in data engineering, with an emphasis on data quality assurance and ETL processes. Required 5 Years
Expertise in Python, PyPI, and SQL Required 5 Years
Demonstrate a strong understanding of cybersecurity principles related to code development, DevOps, data access, and fundamental cybersecurity. Required 5 Years
Understanding of fundamental public-cloud capabilities. Required 5 Years
Proven capacity to comprehend business needs and convert them into technical requirements. Required 5 Years
Job Description:
Quality Assurance & Quality Checking: Implement and maintain data quality to ensure the accuracy and reliability of data throughout the ETL process.
ETL Processes: Design, develop, and optimize ETL workflows to efficiently transfer data from file transfer services to S3 buckets and Snowflake.
Data Integration: Ensure seamless data integration into data platform, enabling efficient consumption by downstream applications and reporting tools.
Data Quality Management: Address data quality challenges, including inconsistencies in source data that do not meet ingestion requirements, which can lead to load failures or data backouts.
Collaboration: Work closely with business owners, data analysts, business intelligence teams, and other stakeholders to understand data requirements and deliver high-quality data solutions.
Monitoring & Troubleshooting: To preserve data flow and integrity, monitor pipelines, identify issues, and implement solutions.
Qualifications (Knowledge/Skills/Abilities):
Demonstrated mid-level experience in data engineering, with an emphasis on data quality assurance and ETL processes.
Expertise in Python, PyPI, and SQL
Expert analytical and problem-solving skills.
Demonstrate a strong understanding of cybersecurity principles related to code development, DevOps, data access, and fundamental cybersecurity.
Understanding of fundamental public-cloud capabilities.
Proven capacity to comprehend business needs and convert them into technical requirements.
Demonstrated excellence in communication and collaboration abilities.
Proven capacity to define success, deliver, and operate in an agile setting.