Summary
Microsoft AI are looking for a talented Relevance Metrics Data or Applied Scientist at their Redmond office. This role sits at the heart of strategic decision-making, turning market data into actionable insights for a company that's revolutionising AI technology. You'll work directly with leadership to shape the company's direction in the AI market.
About the Role
As a Relevance Metrics Data or Applied Scientist, you will be responsible for designing and implementing metrics for Relevance and Accuracy Guidance (RAG) with Bing Web Search and other APIs. You will build pipelines and dashboards for Bing Grounding quality, use Large Language Models (LLMs) in LLM-as-a-judge settings for data evaluation, engineer prompts for textual and multi-model LLMs for data processing and generation of insights, and design and implement End-to-End (E2E) pipelines for data analysis and processing.
Accountabilities
- Design and implement metrics for RAG with Bing Web Search and other APIs.
- Build pipelines and dashboards for Bing Grounding quality.
- Use LLMs in LLM-as-a-judge settings for data evaluation.
- Engineer prompts for textual and multi-model LLMs for data processing and generation of insights.
- Design and implement E2E pipelines for data analysis and processing.
The Candidate we're looking for
Experience:
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Master’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Bachelor’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience.
Technical skills:
- 3 years of T-SQL.
- Experience or deep interest in RAG, Large-Language Models.
- Passion for metrics building for complex multi-step systems.
- Passion for prompt engineering and text generations with LLMs.
- Interest in designing dashboards for data visualization in novel ways.
Personal attributes:
- Ability to work independently, solid collaboration and communication skills.
Benefits
- Competitive salary.
- Comprehensive benefits package.
- Opportunities for professional growth and development.
- Collaborative and dynamic work environment.
- Recognition and rewards for outstanding performance.
