Job Description
Key Responsibilities
- ETL Automation Frameworks: Design, develop, and maintain scalable ETL test automation frameworks to validate data pipelines across ingestion, transformation, and load layers.
- Data Validation Automation: Automate validation of source-to-target mappings, business rules, transformations, and aggregations across large datasets.
- Data Quality & Reconciliation: Perform automated data quality checks, reconciliation testing, and regression testing to ensure data accuracy, completeness, and consistency.
- Databricks & Spark Testing: Validate Databricks notebooks, Spark jobs, workflows, and Delta tables, including batch and incremental processing.
- SQL-Based Validation: Write and optimize complex SQL queries for data validation, anomaly detection, and reconciliation across data sources.
- Python / PySpark Automation: Develop automation scripts using Python and PySpark for validating large-scale data processing logic.
Ready to Apply?
Take the next step in your AI career. Submit your application to Winfo Solutions today.
Submit Application