Machine Learning Engineer, Evaluation

Own and scale Responsible AI evaluation for global generative AI products at Apple, turning safety and inclusivity policy into measurable metrics and repeatable pipelines.

Responsibilities

• Develop evaluation metrics for safety and fairness risks in generative models and Gen-AI features
• Design and evolve datasets, identifying data needs and scaling coverage using human and synthetic generation methods
• Create sampling strategies and combine human annotation with auto-grading to produce high-confidence insights at scale
• Use and implement data pipelines and collaborate cross-functionally to execute end-to-end safety evaluations
• Improve operational efficiency by automating manual workflows, optimizing evaluation velocity, and scaling evaluation infrastructure for rapid release cycles
• Distill evaluation findings into insights for product engineering teams and safety policy development
• Work with highly sensitive content, including exposure to offensive and controversial material

Requirements

• Experience working in the Responsible AI space
• Strong interest in building innovative, high-impact products and teams using interdisciplinary skills
• Curiosity about fairness and bias in generative AI, with motivation to improve equity
• Social Science or Humanities background, or demonstrable interest in socio-technical issues
• Prior scientific research and publication experience
• Proven record contributing to diverse teams in collaborative environments
• Experience with multimodal models, agentic systems, and/or diverse languages and locales
• Experience with compliance and regulatory needs in AI, specifically related to child safety, clinical/health applications, or companion chatbots
• Experience conducting large-scale benchmarking across models and capabilities
• MS or PhD (fields listed: Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field) with a strong basis in scientific data collection and analysis, plus at least 4 years relevant work experience; or BA/BS with 8+ years relevant work experience, including 2+ years directly managing engineering teams
• Experience gathering and analyzing varied language, image, voice, video, and/or multi-modal data, including human- and LLM-generated data
• Experience with LLM-Judges and human-in-the-loop techniques, including designing human annotation projects, writing guidelines, and handling highly multi-labeled, nuanced, often conflicting data
• Proficiency in data science, machine learning, analytics, programming with Python & Pandas, and AI-assisted coding tools; experience with one or more plotting and visualization libraries
• Cross-functional collaboration skills to prioritize competing needs, set and maintain milestone schedules, and communicate with team members and internal/external stakeholders
• Rigorous approach to metric development, interpretation of experiments and evaluations, and presentation to executives
• Comfort leading a team working with sensitive or potentially offensive content

Technologies

• Python
• Pandas

Benefits

• Comprehensive medical and dental coverage
• Retirement benefits
• Range of discounted products and free services
• Reimbursement for certain educational expenses, including tuition
• Discretionary bonuses or commission payments and relocation (if eligible)
• Apple shareholder opportunity through participation in Apple’s discretionary employee stock programs
• Eligible for discretionary restricted stock unit awards
• Can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan

Pay & Benefits Notes

• Base pay range: $184,700 - $324,800 (yearly)
• Base pay determined within the range based on skills, qualifications, experience, and location
• Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of applicable plans or programs
• Learn more about Apple Benefits