Job Description
Job Role: Senior Data Engineer
Experience Level : 3.5 to 10 years
Work location: Bangalore
Joining timeline: 0 to 45 days
Job Description:
Must have skills:
● 3+ years of hands-on experience in building large-scale data pipelines.
● Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
● Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
● Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.
● Solid understanding of AWS services including S3, Redshift, Glue, Cloud Watch, and Secrets Manager
● Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)
● Ability to implement monitoring dashboards and observability for pipelines using tools like Cloud Watch or Datadog
● Strong SQL skills for validating data accuracy and developing job-level metrics
● Expe...
Experience Level : 3.5 to 10 years
Work location: Bangalore
Joining timeline: 0 to 45 days
Job Description:
Must have skills:
● 3+ years of hands-on experience in building large-scale data pipelines.
● Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
● Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
● Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.
● Solid understanding of AWS services including S3, Redshift, Glue, Cloud Watch, and Secrets Manager
● Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)
● Ability to implement monitoring dashboards and observability for pipelines using tools like Cloud Watch or Datadog
● Strong SQL skills for validating data accuracy and developing job-level metrics
● Expe...
Ready to Apply?
Take the next step in your AI career. Submit your application to Quantiphi today.
Submit Application