AI Operations & Quality Engineer

Agent Quality Assurance

• Run regression tests against canonical scenarios for every production agent — catch drift, broken outputs, and edge cases before brokers see them.
• Design and calibrate evaluation rubrics for each agent's deliverables and analyze score patterns to flag systemic issues.
• Reproduce failed runs surfaced by brokers, root-cause them, and ship fixes via memory updates, skill edits, or prompt revisions.
• Audit branded deliverables for brand compliance, verifying compliance across fonts, logos, layouts, and data structure.

Skills, Memories, And Prompts

• Curate the memory library inside Synapse: dedupe, retire stale facts, tune importance and when to use routing so agents pull the right context.
• Author new skills (documented playbooks + Python scripts) that turn repeatable workflows into reusable agent capabilities, with credential handling an...