Data Platform Developer

Job Title: Data Platform Developer

Key Responsibilities

As a Data Platform Developer, you will:

• Solution Design & Development: Design, build, and unit test applications on the Spark framework using Python (PySpark). Translate requirements into full-fledged and scalable PySpark-based applications for both batch and streaming requirements.
• Data Pipeline Development: Develop and execute data pipeline testing processes, and validate business rules and policies. Build integrated solutions leveraging Unix shell scripting, RDBMS, Hive, HDFS File System, HDFS File Types, and HDFS compression codecs.
• Big Data Ecosystem Management: Apply in-depth knowledge of various Hadoop and NoSQL databases.
• Automation & CI/CD: Create and maintain integration and regression testing frameworks on Jenkins, integrated with Bitbucket and/or GIT repositories.
• Agile Collaboration...