Job Description
Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks.
Implement robust data ingestion workflows from multiple sources, including Azure Blob Storage, Azure Data Lake, on-premises SQL databases, and SFTP servers.
Design, transform, and optimize data models using Oracle, Spark SQL, PySpark, SQL Server, and Progress DB SQL.
Build and manage orchestration workflows in ADF utilizing activities such as Lookup, ForEach, Execute Pipeline, and Set Variable.
Conduct root cause analysis and troubleshoot production issues across data pipelines and Databricks notebooks.
Collaborate in building and maintaining CI/CD pipelines using Azure DevOps and Jenkins.
Apply performance optimization and tuning strategies for Azure Synapse Analytics and SQL Data Warehouse (SQL DW).
Maintain comprehensive documentation, including runbooks, technical design specifications, and QA test cases.
Implement robust data ingestion workflows from multiple sources, including Azure Blob Storage, Azure Data Lake, on-premises SQL databases, and SFTP servers.
Design, transform, and optimize data models using Oracle, Spark SQL, PySpark, SQL Server, and Progress DB SQL.
Build and manage orchestration workflows in ADF utilizing activities such as Lookup, ForEach, Execute Pipeline, and Set Variable.
Conduct root cause analysis and troubleshoot production issues across data pipelines and Databricks notebooks.
Collaborate in building and maintaining CI/CD pipelines using Azure DevOps and Jenkins.
Apply performance optimization and tuning strategies for Azure Synapse Analytics and SQL Data Warehouse (SQL DW).
Maintain comprehensive documentation, including runbooks, technical design specifications, and QA test cases.
Ready to Apply?
Take the next step in your AI career. Submit your application to J&M Group today.
Submit Application