Software Data Engineer

Position Description:

Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark.
•Develop data processing solutions using Python and Spark for large-scale data transformation.
• Write complex SQL queries for data extraction, transformation, validation, and reporting.
• Build and optimize data workflows to improve performance, scalability, and reliability.
• Work with structured and semi-structured data from multiple data sources.
• Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
• Perform data quality checks, troubleshooting, and root cause analysis for data issues.
• Optimize Spark jobs and SQL queries for improved performance.
• Follow coding standards, documentation, and best practices for data engineering.
• Participate in code reviews and support production deployments.

Responsibilities:
• Minimum 3 years of experience in D...