Job Description

Job Description:

  • Design, develop, and maintain ETL pipelines using Informatica, Azure Data Factory (ADF), and Databricks
  • Perform migration of legacy ETL workflows (Informatica) to Databricks using Python/PySpark
  • Analyze existing ETL workflows and re-engineer into optimized Spark-based transformations
  • Develop data processing and transformation solutions using Python and PySpark
  • Apply AI/ML techniques for data enrichment, anomaly detection, and predictive insights (where applicable)
  • Build and optimize SQL queries, data models, and transformations
  • Schedule and monitor jobs using AutoSys
  • Integrate data from multiple sources: Relational databases (SQL Server, DB2), Files (CSV, XML, JSON) and mainframe systems
  • Streaming platforms like Kafka
  • Perform data validation, reconciliation, and ensure data quality
  • Troubleshoot ETL/pipeline failures and optimize performance
  • Collaborate...

Ready to Apply?

Take the next step in your AI career. Submit your application to Confidential today.

Submit Application