Model Behavior Architect

At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity.

As a Model Behavior Architect, you are at the forefront of defining and measuring LLM behaviour. We are looking for people who have built a career in engineering, machine learning, and large language models and are experts in model evaluation, policy writing, and creating eval pipelines for complicated tasks.

What you will do

• Interact with models to identify where model behavior can be improved
• Gather internal and external feedback on model behavior to scope areas for improvement
• Design and implement evals, data guidelines, data generation, and synthetic testing environments
• Identify and fix edge case behaviors through rigorous testing
• Develop robust evaluation pipelines for our model candidates
• Work collaboratively with AI Scientists

About you

• You have a deep understanding of either 1) linguistics, language, and translation, 2) engineering and code behavior, 3) LLM agents at work, including reasoning and tool use
• You have prior knowledge in training and optimising model behaviour
• You are an expert at building robust evaluations
• You thrive in dynamic and technically complex environments
• You have a track record of delivering innovative, out-of-the-box solutions to address real-world constraints