Senior Big Data Python Engineer

Let's make things Fly
Your Role & Responsibility:

• Data Transformation and Processing: Extract, transform, and load (ETL) Flight Data Monitoring data using Spark and PySpark, applying relevant algorithms based on engineering requirements to ensure accuracy and consistency.

• Algorithm Implementation: Develop, code, and test PySpark algorithms that meet specific definitions and requirements provided by aircraft engineers, addressing use cases such as anomaly detection, event recognition, and trend analysis in flight data.

• Timeseries Data Management: Structure and optimize large time-series datasets in PostgreSQL Timeseries, with a focus on efficient storage, retrieval, and processing for real-time insights.

• Platform Management: Maintain data processing pipelines within Azure, ensuring scalability, performance, and data security for big data workloads.

• API and Data Delivery: Publish processed data vi...