Big Data Engineer

Key Skills: GCP, GCP Data stack, Cloud (AWS / Azure / GCP), Data Engineer

Roles and Responsibilities:

• Design, develop, and maintain batch and streaming data pipelines on Google Cloud Platform.
• Build, optimize, and manage BigQuery datasets, tables, views, and complex SQL queries.
• Develop data pipelines using Dataflow (Apache Beam), Pub/Sub, and Cloud Storage.
• Perform data ingestion from diverse data sources including APIs, databases, and files.
• Ensure data accuracy, consistency, validation, monitoring, and robust error handling.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Implement IAM, security controls, and compliance standards across data platforms.
• Automate and schedule workflows using Cloud Composer (Airflow).
• Collaborate with business and technical stakeholders to translate requirements into scalable data solutions.
...