AVP/VP, AI/ML Model Validation Engineer, Data Management Office

Responsibilities Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models. Perform functional, regression and scenario‑based testing of model behaviours and workflows. Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection. Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning. Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks. Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues. Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly. Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours. Perform data‑management tasks to support AI/ML model testi...