Data Engineer / 1

Join our team to leverage your data engineering skills in a dynamic environment, ensuring seamless data migration and optimization for advanced AI and ML projects. Apply now to be part of our innovative journey!

Key responsibilities

Data pipeline development:

• Design, develop, and deploy Python-based ETL/ELT pipelines to migrate data from the on-premises MS SQL Server into the Databricks instance,
• Ensure efficient ingestion of historical parquet datasets into Databricks.

Data quality & validation:

• Implement validation, reconciliation, and quality assurance checks to ensure accuracy and completeness of migrated data,
• Handle schema mapping, field transformations, and metadata enrichment to standardize datasets,
• Ensure data governance, quality assurance, and compliance are integral to all migration activities.

Performan...