Python Pyspark

Job description

Senior Data Engineer

Position Summary

The Senior Data Engineer leads complex data engineering projects working on designing data architectures that align with business requirements This role focuses on optimizing data workflows managing data pipelines and ensuring the smooth operation of data systems

Minimum Qualifications

8 Years overall IT experience with minimum 5 years of work experience in below tech skills

Tech Skill

Strong experience in Python Scripting and PySpark for data processing

Proficiency in SQL dealing with big data over Informatica ETL

Proven experience in Data quality and data optimization of data lake in Iceberg format with strong understanding of architecture

Experience in AWS Glue jobs

Experience in AWS cloud platform and its data services S3 Redshift Lambda EMR Airflow Postgres SNS Event bridge

Expertise in BASH Shell scripting

Strong und...