Job Description

Overview

Job Role : Databricks Data Platform Engineer

Job Location : Abu Dhabi, UAE

Experience : 5+ Years

Role Summary : Design and build scalable, high-performance, and governed data pipelines on Azure Databricks to support analytics, reporting, and enterprise data intelligence initiatives.

Responsibilities
  • Key Responsibilities – Develop batch pipelines using Delta Lake with proper table design, partitioning, optimization, and lifecycle management
  • Implement incremental ingestion using Autoloader with schema evolution and checkpointing
  • Build Structured Streaming pipelines with watermarking, state management, and late data handling
  • Develop declarative pipelines using Lakeflow standards
  • Ensure pipelines support idempotency, replayability, and safe backfills
  • Optimize Spark workloads (AQE, skew handling, shuffle tuning, join optimization)
  • ...

Ready to Apply?

Take the next step in your AI career. Submit your application to Epergne Solutions today.

Submit Application