Job Description
Description
As a Data Engineer, the candidate will design and maintain scalable data pipelines and analytics systems. The ideal candidate will have 2- 4 years of experience with Apache Spark,Scala/Python, Trino/Presto, Hadoop,kafka and data lake technologies such as Delta Lake.
Experience with Elasticsearch, streaming data, and modern analytics platforms is preferred.
Mandatory Skills Requirements
As a Data Engineer, the candidate will design and maintain scalable data pipelines and analytics systems. The ideal candidate will have 2- 4 years of experience with Apache Spark,Scala/Python, Trino/Presto, Hadoop,kafka and data lake technologies such as Delta Lake.
Experience with Elasticsearch, streaming data, and modern analytics platforms is preferred.
Mandatory Skills Requirements
- Proficient in Python and/or Scala with strong experience in developing and optimizing data processing applications using Apache Spark.
- Extensive experience with Apache Spark Structured Streaming for near real-time and streaming data processing.
- Strong hands-on experience with Apache Kafka, including integration with Spark for reliable real-time data ingestion and event-driven pipelines.
- Experience working with analytical and distributed data stores such as ClickHouse, Trino/Presto, and da...
Ready to Apply?
Take the next step in your AI career. Submit your application to Octro Inc. today.
Submit Application