Databricks/ PySpark Developer

Looking for an offshore Lead Databricks/ PySpark Developer who is willing to learn new technologies if needed and able to work with a team.

Essential Job Functions:

• Design and development of data ingestion pipelines (Databricks background preferred).
• Performance tune and optimize the data bricks jobs
• Evaluated new features and refractors existing code
• Mentor junior developers and makes sure all patterns are documented
• Perform data migration and conversion activities.
• Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures.
• Collaborate with Business Analysts, Architects and Senior Developers to establish the physical application framework (e.g. Libraries, modules, execution environments).
• Perform end to end automation of ETL...