Data Engineer - Pyspark, Airflow, AWS

Key Responsibilities:

● Develop Apache Airflow DAGs and PySpark ETL pipelines for high volume data processing.

● Write optimized SQL queries for data transformation and aggregation.

● Build data products serving Business Process, Executive KPIs, and Product Analytics.

● Implement data quality and monitoring solutions.

● Optimize pipeline performance and troubleshoot production issues.

● Collaborate with cross-functional teams.

● Production Pipeline Monitoring (KLO).

Required skills:

● 10+ years of data engineering experience. 7+ years dedicated to the Big data stack.

● Expert in Python and PySpark (DataFrame API, Spark SQL).

● Advanced SQL skills (w...