Python and PySpark ETL Developer

Python, PySpark, ETL Developer Technology->Analytics - Packages->Python - Big Data,Technology->Big Data - Data Processing->PySpark, ETL • Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent practical experience). • 2–5 years of hands-on experience building data pipelines using Python and PySpark. • Strong understanding of ETL concepts, data transformations, and handling large-scale datasets. • Proficiency in writing clean, maintainable code and debugging production issues. • Working knowledge of data structures, algorithms, and software development best practices. Data Pipeline Development • Develop and maintain scalable batch ETL pipelines using Python and PySpark for data ingestion, transformation, and loading. • Implement reusable transformation logic, ensuring pipelines are modular, testable, and easy to maintain. • Optimize Spark jobs for performance (partitioning, caching, joins, shuffles) and cost efficiency. Data Quality & ...