Job Description
About Us
MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.
Job Title: AI Engineer
Experience: 5+ Years
Working Model: Remote
Responsibilities
Build Python-based pipelines for automated quality testing of AI responses.
Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
Automate regression and stress testing for conversational AI flows.
Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
Implement both rule-based and AI-driven quality checks.
Monitor model drift, bias, and hallucinations using automated workflows.
Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
Develop monitoring dashboards to visualize conv...
MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.
Job Title: AI Engineer
Experience: 5+ Years
Working Model: Remote
Responsibilities
Build Python-based pipelines for automated quality testing of AI responses.
Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
Automate regression and stress testing for conversational AI flows.
Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
Implement both rule-based and AI-driven quality checks.
Monitor model drift, bias, and hallucinations using automated workflows.
Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
Develop monitoring dashboards to visualize conv...
Ready to Apply?
Take the next step in your AI career. Submit your application to MyRemoteTeam Inc today.
Submit Application