Senior Data Scientist - Outlook Science Team

Lead the design and execution of analytical frameworks that evaluate product investments across Outlook Mail, Calendar, Search, and Copilot experiences. Define metrics, value-creating interactions, guardrails, funnels, retention measures, and feature-level success criteria in partnership with product managers, engineers, and scientists. Design and analyze online experiments, quasi-experiments, and observational studies to estimate causal impact and guide ship decisions. Develop trusted dashboards, semantic models, and reporting pipelines that monitor product health, experimentation outcomes, AI quality, reliability, latency, sentiment, and adoption. Evaluate generative AI and agentic experiences using offline and online evaluation methods, including benchmark datasets, human judgment, LLM-as-a-judge approaches, error analysis, and production telemetry. Apply prompt engineering and evaluation techniques to improve response quality, grounding, relevance, safety, and task completion for Outlook scenarios. Partner with applied scientists and engineers on model selection, feature development, post-training signals, and measurement strategies for AI-powered product experiences. Use large-scale behavioral and telemetry data to identify customer needs, diagnose product issues, forecast trends, and uncover opportunities across commercial and consumer segments. Influence product strategy by communicating clear, actionable insights to senior leaders and translating ambiguous business questions into measurable analytical plans. Provide technical mentorship and raise the bar for statistical rigor, reproducibility, privacy-conscious analytics, and operational excellence across the data science community. Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience. These requirements include but are not limited to the following specialized security screenings: Experience with Generative AI technologies, such as large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, and natural language processing (NLP), with an understanding of AI quality dimensions such as relevance, groundedness, factuality, safety, reliability, latency, and user trust. Hands-on experience in AI evaluation and optimization, such as prompt engineering, benchmark and rubric design, LLM evaluation, human-in-the-loop assessment, A/B testing, offline evaluations, production telemetry analysis, and leveraging customer feedback to improve AI product performance. Experience working with large, distributed datasets and building reusable analytical datasets, metrics layers, dashboards, or self-service analytics solutions. Advanced proficiency with big-data technologies and cloud analytics platforms. Experience with Azure Data Explorer/Kusto, Azure Data Lake, Synapse, Databricks, Data Factory, or Power BI is a plus. Experience driving product analytics for communication, productivity, search, collaboration, or enterprise software products, with a proven ability to lead ambiguous cross-functional initiatives, influence stakeholders, communicate insights effectively, and deliver measurable business impact at scale. Advanced degree in a quantitative or technical field is preferred. 6+ years of experience in data science, statistics, machine learning, product analytics, business intelligence, or data-driven product strategy. 4+ years of experience using SQL and Python or R to conduct analysis and implement statistical or machine learning methods, such as regression, classification, clustering, forecasting, causal inference, or experimentation. Experience defining product metrics and using large-scale data to influence roadmap, investment, and ship decisions. Experience designing experiments and applying statistical methods to measure product or business impact.