Data Engineer

Overview

Design, build, and maintain cloud-based data pipelines and workflows that support analytics and operational systems. Integrate data from various sources using APIs and cloud services. Develop clean, efficient, and test-driven code in Python for data ingestion and processing. Optimize data storage and retrieval using big data formats like Apache Parquet and ORC. Implement robust data models, including relational, dimensional, and NoSQL models. Collaborate with cross-functional teams to gather and refine requirements and deliver high-quality solutions. Deploy infrastructure using Infrastructure as Code (IaC) tools such as AWS CloudFormation or CDK. Monitor and orchestrate workflows using Apache Airflow or Dagster. Follow best practices in data governance, quality, and security.

Responsibilities

• Design, build, and maintain cloud-based data pipelines and workflows to support analytics and operational systems.
• Integrate data from various...