Data Engineering Architect (ATC)

Experience:

• 8+ years of experience in data engineering, specifically in cloud environments like AWS.
• Proficiency in PySpark for distributed data processing and transformation.
• Solid experience with AWS Glue for ETL jobs and managing data workflows.
• Hands-on experience with AWS Data Pipeline (DPL) for workflow orchestration.
• Strong experience with AWS services such as S3, Lambda, Redshift, RDS, and EC2.

Technical Skills:

• Proficiency in Python and PySpark for data processing and transformation tasks.
• Deep understanding of ETL concepts and best practices.
• Familiarity with AWS Glue (ETL jobs, Data Catalog, and Crawlers).
• Experience building and maintaining data pipelines with AWS Data Pipeline or similar orchestration tools.
• Familiarity with AWS S3 for data storage and management, including file formats (CSV, Parquet, Avro).
• Strong knowledge ...