Data Engineer

About the Role

• Build and improve large-scale data platforms that support an AI-powered SaaS product
• Work with tens of terabytes of enterprise data used by real business users
• Design data foundations for future AI / data science use cases
• Work on a proven product with real users and real impact

Responsibilities

• Design, build, and improve big data pipelines for a SaaS + database product
• Develop and maintain data pipelines for:
• Web data crawling
• Data cleansing and enrichment
• Build data quality monitoring and validation systems
• Improve scalability, performance, and cost efficiency of data infrastructure
• Design data platforms and data quality foundations with future analytics and AI usage in mind

Tech Stack

Data Ingestion

• Python (Scrapy)
• AWS Fargate (Amazon ECS)
• Amazon Managed Workflows for Apache Airflow

D...