Job Description
Elevate AI quality assurance with Summit as an AI Operations Engineer. Focus on regression testing, prompt engineering, and optimizing AI agent performance in a flexible hybrid work environment.
Join Summit in their innovative pursuit of AI excellence. This role requires 2-4 years of hands-on experience with LLM-powered applications and a strong grasp of prompt engineering. You'll design evaluation rubrics, troubleshoot agent issues, and ensure compliance across deliverables while collaborating closely with broker teams.
Key Responsibilities:
• Run regression tests to catch drift and edge cases
• Design evaluation rubrics for agent deliverables
• Reproduce and fix failed runs reported by brokers
• Monitor model spend and optimize selection
• Document agent capabilities and workflows for brokers
Requirements:
• 2-4 years with AI agents or prompt-engineering systems
• Proficiency in Python for scripting tasks
• Experience in designing test cases for non-d...
Join Summit in their innovative pursuit of AI excellence. This role requires 2-4 years of hands-on experience with LLM-powered applications and a strong grasp of prompt engineering. You'll design evaluation rubrics, troubleshoot agent issues, and ensure compliance across deliverables while collaborating closely with broker teams.
Key Responsibilities:
• Run regression tests to catch drift and edge cases
• Design evaluation rubrics for agent deliverables
• Reproduce and fix failed runs reported by brokers
• Monitor model spend and optimize selection
• Document agent capabilities and workflows for brokers
Requirements:
• 2-4 years with AI agents or prompt-engineering systems
• Proficiency in Python for scripting tasks
• Experience in designing test cases for non-d...
Ready to Apply?
Take the next step in your AI career. Submit your application to Summit today.
Submit Application