Job ID: MNSITE-2384 (90090506)3P
Remote Data Lake Developer with ETL, API, ETL/ELT, Azure Synapse Pipeline/Data Factory, Apache Spark Pools, python, Azure DevOps/Git Repos/Data Studio/Analytics, Azure SQL Server, Parquet, Power BI and Agile experience
Location: St Paul MN (MNIT)
Duration: 6 months
Attachments: RTR/Skill Matrix
Minimum Qualifications
•Five years of Data Lake Developer experience
•Two engagements with two different entities in a Senior Data Lake Developer role
Preferred Qualifications
•Experience in data lake configuration, setup, and bringing in data from various data sources using ETL, API
•Experience in Azure Cloud services and solutions
•Experience working with enterprise data warehouse
•Experience as an ETL/ELT Developer using various ETL/ELT tools such as Azure Synapse Pipelines and Azure Data Factory, Apache Spark Pools using python scripts
•Experience in Azure DevOps Services using Azure Git Repos, Azure ¬data studio, Azure Analytics, data mapping, deployment artifacts and release packages for test & production environment
•Experience in building end-end scalable data solution, from sourcing raw data, transforming data to producing analytics reports
•Experience in Python (ETL and Data Visualization libraries)
•Experience in Azure SQL databases across SQL DB, Managed instance & Data warehouse
•Experience in Azure platform services such as blob storage, event hubs, monitoring services
•Experience in creating data structures optimized for storage and various query patterns, for example Parquet
•Experience in building secured Power BI reports, dashboards, paginated reports
•Experience in working in an Agile SDLC methodology
Sample Tasks
•Analyze and define data requirements
•Establish automated data extraction process for Sciforma, SWIFT and other PPM tools (i.e., API or other preferred methods)
•Develop data lake structure and populate with data
•Develop reports/queries for advanced data analytics
•Develop Power BI reports/data visualization using advanced queries
•Provide knowledge transfer
Description of Project
The Minnesota Department of Information Technology Services (MNIT), through its Transformation Office, is seeking to build a data lake repository of project and portfolio management (PPM) data for the MNIT enterprise to enable the organization to make data-driven decisions for its IT projects. For this project, MNIT is seeking a senior-level data lake developer. The database will be used as a single source of truth for both enterprise and MNIT partner agency reporting needs, along with making data available for advanced analytics purposes. The work will be done in phases. The pilot phase will be focused on getting data from Sciforma and the statewide integrated financial tool (SWIFT) (project financials). The long term goal is to capture data from all PPM tools being used throughout MNIT’s partner agencies in the data lake. Additional phases may be added via amendment to the work order.