Data Engineer

Job Responsibilities:

Responsible for designing, developing, and maintaining data pipelines and ETL processes

using Databricks and Azure Data Factory.

Good experience in using SQL, Python and Spark within Databricks notebooks for data

processing and transformations.

Integrate and orchestrate data from various sources, ensuring data quality and accuracy.

Build reusable rules for Data Quality and required transformations.

Optimize, monitor data pipelines for performance and scalability and debug in case of issues.

Proficiency in Databricks, Azure Data Factory, Python, SQL is must.

Familiarity with ETL principles, data warehousing, and big data technologies.

Evaluate and define technical specs using BRD for BI and DW solutions.

Build conceptual and logical models based on the functional flow of business in scalable way.

Work with Business functions/ leadership and Application SMEs to understand the

...