Job Description
Exigo Tech is a Sydney-based Technology Solutions Provider that is focused on providing solutions on three major verticals; Infrastructure, Cloud, and Application to businesses across Australia. We help companies reach operational efficiencies by empowering them with technology solutions that drive their business processes.
Exigo is looking for Full-time Sr. Data Engineer
Click Here to know more : LIFE AT EXIGO TECH
Roles and Responsibilities
Install, configure, and manage Apache Spark (open-source) clusters on Ubuntu, including Spark master/worker nodes and Spark environment files.
Configure and manage Spark UI and Spark History Server for monitoring jobs, analyzing DAGs, stages, tasks, and troubleshooting performance.
Develop, optimize, and deploy PySpark ETL/ELT pipelines using DataFrame API, UDFs, window functions, caching, partitioning, and broadcasting.
Deploy PySpark jobs using spark-submit in client/cluster mode with proper logging and err...
Exigo is looking for Full-time Sr. Data Engineer
Click Here to know more : LIFE AT EXIGO TECH
Roles and Responsibilities
Install, configure, and manage Apache Spark (open-source) clusters on Ubuntu, including Spark master/worker nodes and Spark environment files.
Configure and manage Spark UI and Spark History Server for monitoring jobs, analyzing DAGs, stages, tasks, and troubleshooting performance.
Develop, optimize, and deploy PySpark ETL/ELT pipelines using DataFrame API, UDFs, window functions, caching, partitioning, and broadcasting.
Deploy PySpark jobs using spark-submit in client/cluster mode with proper logging and err...
Ready to Apply?
Take the next step in your AI career. Submit your application to Exigo Tech today.
Submit Application