DataJobs.io
← Back to all jobs

Job Description

Meta’s Safety team within MSL (Meta Superintelligence Labs) is hiring a Data Scientist to develop measurement, evaluation, and monitoring capabilities that support policy enforcement and address emergent harms. This role combines statistical telemetry, experiment design, safeguards evaluation, and real-time reporting to inform safety interventions and long-term roadmaps.

Role Overview

In this position, you will build statistical systems to measure policy violations and emergent harms, design evaluation frameworks that keep pace with evolving agentic model capabilities, and run experiments to test policy and risk definition changes. You will also develop intelligence pipelines, signals, and dashboards that enable real-time monitoring and actionable safety decisions.

Key Responsibilities

  • Measure abuse: develop statistical telemetry and measurement frameworks to detect and monitor policy violations or emergent harms
  • Scale model evaluation: design a scalable evaluation framework aligned with changing agentic model capabilities and the evolving safety and risk landscape, using statistical foundations
  • Evaluate and optimize safeguards: scale offline and online performance evaluation of safeguards, including human-in-the-loop and active learning
  • Design experiments and analyses: conduct controlled experiments and rollouts to evaluate the impact of policy or risk definition changes and safety mitigations
  • Build intelligence and reporting: create signals, pipelines, and dashboards for real-time monitoring that supports safety interventions and safety roadmaps

Required Qualifications

  • Bachelor’s degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience
  • Additional option: Bachelor’s degree in Mathematics, Statistics, Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience
  • Minimum 6 years of work experience in analytics (minimum 4 years with a Ph.D.)
  • Experience with data querying languages such as SQL, scripting languages such as Python, and/or statistical or mathematical software such as R

Preferred Qualifications

  • Experience in frontier AI products or risks, including work in Trust & Safety or Fraud/Risk/Security domains in online, adversarial environments; familiarity with model evals, threat modelling, actor telemetry, and human-in-the-loop review systems
  • Ability to operate in fast-paced, high-ambiguity, cross-functional environments, translating technical risk insights into plain language for policy stakeholders
  • Background working with ambiguous and sparse data to operationalize harm prevalence measurement, causal inference, and root-cause analysis with strong quantitative and statistical foundations
  • Master’s or Ph.D. degree in a quantitative field
  • Experience implementing responsible, ethical AI practices such as risk assessment, bias mitigation, and quality and accuracy reviews
  • Proven ability to integrate AI tools to optimize or redesign workflows and drive measurable impact (for example, efficiency gains and quality improvements)
  • Demonstrated ongoing AI skill development such as prompt/context engineering and agent orchestration, along with staying current with emerging AI technologies

Technologies

  • SQL
  • Python
  • R

Location and Compensation

  • Location: San Francisco, CA (onsite)
  • Compensation: USD 177,000 - 247,000 per year

Similar Jobs