Data Scientist Specializing in Model Monitoring (Toronto)

Overview

Become a key contributor as a Data Scientist specializing in ML and LLM model monitoring and validation. Your expertise will help uphold the integrity and performance of our machine learning systems.

As a Data Scientist reporting to the Director, youll focus on model validation and build critical evaluation frameworks. You’ll engage in designing custom metrics and regression test suites while collaborating with MLOps teams to ensure model performance remains consistent. Experience in Python and SQL will be essential for overseeing various aspects of model evaluations.

Key Responsibilities

• Design and implement ML model evaluation frameworks
• Build regression test suites to identify unexpected behaviors
• Conduct MCP evaluations to assess model reliability
• Monitor and address model performance issues proactively
• Clearly document and present evaluation methodologies

Qualification...