As a Staff Machine Learning Research Scientist on the LLM Evals team, you will lead the development of novel evaluation methodologies, metrics, and benchmarks to measure the capabilities and limitations of frontier LLMs.
You will drive research on the effectiveness and limitations of existing LLM evaluation techniques, design and develop novel evaluation benchmarks for large language models, and communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects.
Key responsibilities include collaborating with internal teams and external partners to refine metrics and create standardized evaluation protocols, implementing scalable and reproducible evaluation pipelines using modern ML frameworks, publishing research findings in top-tier AI conferences, and mentoring and guiding research scientists and engineers.
Ideally, you'd have 5+ years of hands-on experience in large language model, NLP, and Transformer modeling, experience and track record of recording major research impacts in a fast-paced environment, and excellent written and verbal communication skills.
Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position, determined by work location and additional factors.
This role may be eligible for additional benefits such as a commuter stipend.
XML job scraping automation by YubHub
