Data Engineer

Job Description
Data Pipeline Development
• Develop and maintain ETL/ELT pipelines in Databricks using Python (PySpark) and SQL
• Build data pipelines aligned to medallion architecture (Bronze, Silver, Gold layers)
• Implement transformations, aggregations, and data enrichment logic to support downstream analytics
• Ensure all pipelines are scalable, efficient, and production-ready
Workflow Migration (Alteryx → Databricks)
• Convert Alteryx workflows into Databricks-native pipelines
• Translate legacy business logic into optimized Spark-based transformations
• Support migration of both:
o Simple workflows (standard transformations)
o Complex workflows (multi-step logic, flat-file dependencies)
• Validate output to ensure high parity with legacy systems
Data Ingestion & Processing
• Build and maintain ingestion pipelines for:
o Structured data sources
o Flat files and semi-structured datasets
• Implement robust ingestion frameworks to h...