Job Description
Overview
We are hiring a Mid–Senior Data Engineer with strong expertise in Azure cloud, Databricks, PySpark, and orchestration of data pipelines. The ideal candidate must hold a minimum of an Honours degree in an IT-related field and be experienced in building scalable, automated data solutions on Azure.
Key Responsibilities
Design, develop, and maintain scalable and automated data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake.
Build and optimize PySpark-based data workflows in Azure Databricks for batch and streaming workloads.
Develop ETL/ELT processes and ensure reliable orchestration and scheduling of data workflows.
Implement CI/CD practices and DevOps methodologies for data pipeline deployments.
Ensure high availability, performance, and security of data pipelines in production environments.
Monitor, troubleshoot, and resolve issues in data workflows and pipeline performance.
Document data pipelines, architectures, processes,...
We are hiring a Mid–Senior Data Engineer with strong expertise in Azure cloud, Databricks, PySpark, and orchestration of data pipelines. The ideal candidate must hold a minimum of an Honours degree in an IT-related field and be experienced in building scalable, automated data solutions on Azure.
Key Responsibilities
Design, develop, and maintain scalable and automated data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake.
Build and optimize PySpark-based data workflows in Azure Databricks for batch and streaming workloads.
Develop ETL/ELT processes and ensure reliable orchestration and scheduling of data workflows.
Implement CI/CD practices and DevOps methodologies for data pipeline deployments.
Ensure high availability, performance, and security of data pipelines in production environments.
Monitor, troubleshoot, and resolve issues in data workflows and pipeline performance.
Document data pipelines, architectures, processes,...
Ready to Apply?
Take the next step in your AI career. Submit your application to FROSTBYTE today.
Submit Application