Machine Learning Engineer, Evaluation - Responsible AI and Safety

Apple is seeking a Senior Machine Learning Engineer to lead Responsible AI evaluation for global generative AI products. The role focuses on assessing, measuring, and improving safety and inclusivity, with an emphasis on building evaluation metrics, datasets, methods, and data pipelines that support product launch decisions.

What You’ll Do

• Operationalize policy and risk assessments into measurable criteria, including designing datasets, developing LLM and human-in-the-loop analysis methods, and implementing robust data pipelines that produce reliable, scalable insights for ship/no-ship decisions.
• Run hands-on evaluation work before and after launch by analyzing results, investigating failures, and using both human and automated feedback to improve generative feature safety and strengthen user trust.
• Maintain technical excellence, enable leadership with data-driven safety recommendations, and work through complex engineering roadblocks.
• Defend safety metrics with stakeholders and decision-makers.

Core Responsibilities in Evaluation Execution

• Design evaluation approaches that quantify safety and inclusivity for generative AI features.
• Use human and automated signals to iterate on evaluation outcomes and decision-making signals.

Requirements

• Develop metrics to evaluate safety and fairness risks in generative models and Gen-AI features.
• Design datasets, determine data needs, and use creative approaches to scale data coverage through human and synthetic generation.
• Define sampling strategies and pair human annotation with auto-grading to produce high-quality, high-confidence insights at scale.
• Design, implement, and optimize data pipelines, collaborating cross-functionally to deliver end-to-end safety evaluations.
• Improve operational excellence by automating manual workflows and increasing evaluation velocity to support rapid product release cycles.
• Translate complex findings into actionable insights for product engineering teams and safety policy development.
• Work with highly sensitive content, including exposure to offensive and controversial material.

Preferred Qualifications

• Experience with multimodal models, agentic systems, and/or diverse languages and locales.
• Experience with compliance and regulatory needs in AI, particularly child safety, clinical/health applications, or companionship chatbots.
• Experience conducting large-scale benchmarking across models and capabilities.

Compensation & Benefits

• Base pay range: USD $165,800 to $292,400 per year, depending on skills, qualifications, experience, and location.
• Opportunity to become an Apple shareholder through Apple’s discretionary employee stock programs, including discretionary restricted stock unit awards, and the ability to purchase Apple stock at a discount via the Employee Stock Purchase Plan.
• Comprehensive medical and dental coverage.
• Retirement benefits.
• A range of discounted products and free services.
• Reimbursement for certain educational expenses, including tuition, for formal education related to advancing your career.
• Eligibility for discretionary bonuses or commission payments and relocation may apply.

Additional Information

Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program. Learn more about Apple Benefits.

Location & Work Mode

Cambridge, MA (onsite).