DataJobs.io
← Back to all jobs

Job Description

Build and ship production classification and evaluation capabilities that translate research needs into reliable, measurable model behavior.

Responsibilities

  • Build, deploy, and operate production classification and evaluation systems that deliver actionable feedback to post-training and research teams
  • Optimize production inference performance, including model serving, batching, caching, throughput, latency, compute utilization, and cost
  • Train and optimize classifiers using supervised fine-tuning and distillation, validating quality and efficiency prior to deployment
  • Own reliability and observability for deployed classifiers over time
  • Design rigorous evaluations to measure accuracy, calibration, generalization, and potential measurement bias to inform production tradeoffs
  • Partner with software engineers to integrate models into reliable execution systems and support safe rollouts
  • Create reusable engineering tools to accelerate and standardize classifier training, evaluation, deployment, and iteration
  • Collaborate with post-training researchers and data scientists to convert needs into modeling objectives, evaluation criteria, and production requirements

Requirements

  • Experience taking ML projects from problem formulation and experimentation through production deployment
  • Strong foundations in supervised learning, statistics, and experimental design
  • Practical experience evaluating models, investigating systematic errors, and working with imperfect labels
  • Strong software engineering skills with the ability to build maintainable production ML systems
  • Experience reasoning about inference performance and the tradeoffs between quality, latency, and compute cost
  • Ability to combine scientific judgment with a practical approach to shipping and operating software
  • Comfort working closely with researchers and engineers on open-ended problems
  • Fluency in using AI agents to move across multiple workstreams in a fast-paced environment

About the Team

  • Build classification and evaluation systems that provide post-training and research teams fast, reliable feedback on model behavior
  • Combine machine learning and production engineering to deliver accurate, efficient capabilities at scale
  • Work closely with research teams to develop useful measurements and make them practical to run
  • Own classifier development, rigorous evaluation, and efficient production inference, from initial experimentation through deployment and continued improvement

About the Role

  • Develop new classification and evaluation capabilities and bring them into production
  • Translate open-ended questions about model behavior into well-defined ML problems, implement methods to address them, and establish reliability
  • Work across traditional machine learning, model evaluation, and real-time engineering
  • Own classifiers through the full lifecycle: supervised fine-tuning, distillation, evaluation, deployment, and production inference
  • Partner with researchers to identify valuable problems and with software engineers to scale promising approaches

Benefits

  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, plus multiple paid coordinated office closures throughout the year for focus and recharge
  • Paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend
  • Daily meals in offices, and meal delivery credits as eligible
  • Relocation support for eligible employees
  • Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided

Compensation

  • USD 381,000 - 510,000 per year + offers equity
  • Base pay may vary based on individualized factors, including performance-related bonus(es) for eligible employees and the benefits listed
  • Role is at-will; OpenAI may modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions

Similar Jobs