Senior Data Analytics Engineer

This role is a platform-focused Senior Data Analytics Engineer position at Microsoft, based in the United States (onsite). The work centers on designing and operating real-time and batch data pipelines, governed semantic models, and serving infrastructure that supports CSS intelligence and detection products.

Role Summary

Own the end-to-end systems that power detection intelligence, including pipeline reliability, data quality and observability, and productionization of AI decision systems. The role emphasizes governed certified measures and traceable query paths rather than reporting dashboards.

Key Responsibilities

• Design and operate data pipelines feeding CSS intelligence, covering batch processing at organizational volume and near real-time streams where detection latency matters.
• Ingest case and agent telemetry, integrate with the unified data platform and Finance systems, and build scoring pipelines that run data science models continuously in production.
• Ensure pipeline reliability through monitoring and alerting, addressing schema drift and late-arriving data, and escalating failures promptly so issues surface before executive review.
• Build and own the semantic layer between raw data and consuming experiences, including dimensional models, certified measures with documented definitions and clear ownership, consistent hierarchies, and security context for data access.
• Support detection engine queries and executive natural-language experiences via a semantic API, enabling governed measures with a traceable query path instead of improvised calculations.
• Embed data quality and observability into pipelines using freshness stamps, reconciliation tests against sources of record, data contract checks, and end-to-end lineage for tracing numbers back to their underlying rows.
• Apply engineering practices to analytics using source control, deployment pipelines, environment separation, and infrastructure defined as code; manage cost and performance through partitioning, incremental refresh, and query optimization.
• Partner with data scientists and applied AI engineers to convert experimental models into production services and prepare data for AI consumption, including grounding, retrieval, and feature-serving patterns used by agents.
• Design for team handoff with legible schemas, APIs, deployment approaches, and documentation for downstream teams.

Required Qualifications

• Education and experience: Master’s Degree in Mathematics, Analytics, Data Science, Engineering, Computer Science, Business, Economics, or a related field and 2+ years of experience in data analysis and reporting, data science, business intelligence, or business and financial analysis; or Bachelor’s Degree in Statistics, Mathematics, Analytics, Data Science, Engineering, Computer Science, Business, Economics, or a related field and 4+ years of experience in data analysis and reporting, data science, business intelligence, or business.
• Hands-on experience with Microsoft Fabric, Azure Synapse, Azure Data Factory, Databricks, or Power BI semantic models in a production environment.
• Experience building data platforms for AI or machine learning workloads, including feature serving, retrieval and grounding data, or model scoring at scale.
• Experience implementing data governance in practice, including certified metric definitions, tiered metric ownership, and promotion processes for executive reporting.
• Experience supporting executive-facing analytics where accuracy, freshness, and traceability are required.

Technologies

• SQL, Python, Spark, PySpark, Microsoft Fabric, Azure Data Factory, Azure Synapse, Databricks, DAX, Power BI
• CI/CD, Infrastructure as code, API, Natural language, Azure

Core Skills and Experience

• Deep SQL expertise and strong Python skills, including distributed processing with Spark or PySpark.
• Experience building production pipelines in batch and streaming or near real-time modes on platforms such as Microsoft Fabric, Azure Data Factory, Synapse, Databricks, or equivalents.
• Strong dimensional and semantic modeling, including star schema design, tabular or semantic models, and calculation languages such as DAX, with discipline around certified single-definition measures.
• Experience with layered architectures that separate raw, conformed, and serving tiers, including knowing which transformations belong in each tier.
• Ability to expose data and measures programmatically through APIs or query services for applications, services, or AI agents, not only reporting tools.
• Data quality and observability practices, including reconciliation testing, data contracts, freshness and completeness monitoring, anomaly detection, and end-to-end lineage.
• Software engineering discipline for data work, including source control, CI/CD, deployment pipelines, environment separation, code review, and infrastructure as code.
• Platform cost and performance optimization through partitioning, incremental refresh, aggregation strategy, capacity management, and query tuning.
• Experience preparing data for AI and machine learning consumption, including feature pipelines, grounding and retrieval patterns, embedding or vector stores, and modeling agent telemetry (identifiers, token usage, evaluation output).
• Collaboration with data scientists to productionize models and with business stakeholders to translate operational questions into durable data designs.
• A platform mindset focused on reusable, documented capability and recognition of when governed measures already answer a question.
• Comfort operating with autonomy on a small team, owning components end to end, and writing documentation so others can run the work.

Examples of Work in This Role

• Build a certified measure layer used consistently by the detection engine, executive brief experiences, natural-language interactions, and monthly review content to prevent surface-level recalculation.
• Design a semantic API and query service that lets an AI agent answer questions against governed measures under the requester’s security context, including freshness and query traceability.
• Stand up continuous scoring pipelines that compute case propensity and complexity models across CSS case volume and deliver results to detection and workflow systems.
• Ingest AI agent telemetry including agent identifiers, model and version, token consumption, and evaluation results, then model the data to compare quality and cost across platforms.
• Deliver detection infrastructure that computes baselines, thresholds, and materiality gates on a schedule and routes validated signals into notification and workflow paths.
• Implement reconciliation and freshness monitoring to automatically catch and attribute discrepancies between the platform and sources of record.
• Establish deployment, versioning, and documentation patterns to hand capabilities to partner engineering teams for broader operation.

Preferred or Additional Qualifications

• Master’s Degree in Mathematics, Analytics, Data Science, Engineering, Computer Science, Business, Economics, or a related field and 6+ years of experience in data analysis and reporting, data science, business intelligence, or business and financial analysis; or Bachelor’s Degree in Statistics, Mathematics, Analytics, Data Science, Engineering, Computer Science, Business, Economics, or a related field and 8+ years of experience in data analysis and reporting, data science, business intelligence, or business.

Location and Compensation

• Location: United States (onsite)
• Salary Range: USD 106,400 - 222,600 per year
• Minimum Experience: 2 years