Evaluation Datasets Jobs

1 job found

Today

Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Apple San Diego, CA

San Diego, CA

Today

$142k - $263k/yr Full time Ai Testing Artificial Intelligence Engineer

Apply Now

Job Description

Apple is seeking a Machine Learning Engineer to lead evaluation and quality efforts for AI features within its Creativity Apps. This role focuses on defining evaluation strategy, building reliable evaluation datasets and infrastructure, and translating quality findings into decisions that affect shipping features for customers.

Key Responsibilities

Own evaluation and quality for AI features across Apple Creativity Apps.
Collaborate with Machine Learning engineering, infrastructure, product, and design teams to understand models and use cases, then convert those needs into evaluation methodology, testing strategy, and quality metrics aligned with creator experiences.
Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the specific model, product context, and use cases.
Curate and create evaluation datasets, assess and improve dataset quality, and run iterative improvement loops to keep datasets representative across products.
Design, build, maintain, and optimize evaluation infrastructure at scale, including reproducible runs, tracked baselines, and comparisons across model versions.
Use ML, LLMs, and agentic workflows within the evaluation process, including automated judging, test generation, and analysis to improve efficiency, accuracy, and scalability.
Drive end-to-end evaluation and quality from model checkpoints to integrated, shipping features, including triaging issues and driving them to resolution.
Report quality clearly by converting evaluation outcomes into ship/no-ship risk and recommendations for both technical and non-technical audiences.
Partner with cross-functional teams and make informed judgments in ambiguous environments.

Required Qualifications

5+ years of industry experience in machine learning or software engineering, focused on ML/AI model evaluation and quality.
Experience applying evaluation methodology to ML models, LLMs, and AI products or features.
Strong understanding of the ML development lifecycle, including data, training, evaluation, deployment, and monitoring.
Solid machine learning fundamentals and LLM knowledge, including agentic workflows, with attention to recent field developments.
Track record applying LLMs or agentic workflows to ML evaluation and quality.
Experience with data curation, dataset creation, and dataset quality, including iterative improvement workflows that produce high-quality, product-specific datasets.
Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale.
Experience designing evaluation metrics based on model behavior, product requirements, and use cases.
Experience driving end-to-end evaluation and quality for AI products.
Fluency in quality reporting, including presenting quality data to both technical and non-technical audiences.
Strong communication skills and demonstrated experience working with and driving cross-functional partners.
Proficiency in Python and/or Swift or comparable languages.
Bachelor’s degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.

Preferred Qualifications

Knowledge of the software development life cycle and software quality assurance methodologies.
Experience with objective image and video quality evaluation and metrics.
Knowledge of color science and/or experience with subjective image or video quality review.
Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus.
Interest in learning new technology and applying it to practical problems.
Familiarity with iOS and macOS.

Tech Stack

Python
Swift
iOS
macOS

Location and Compensation

Location: San Diego, CA (onsite)

Compensation: USD 142,300 - 263,300 per year

The base pay range is determined based on skills, qualifications, experience, and location.

Benefits

Comprehensive medical and dental coverage
Retirement benefits
Discounted products and free services
Reimbursement for certain educational expenses (including tuition) for formal education related to advancing your career
Opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs
Discretionary restricted stock unit awards
Ability to purchase Apple stock at a discount if voluntarily participating in the Employee Stock Purchase Plan
Discretionary bonuses or commission payments, and relocation may be eligible

View full job page →

Get Evaluation Datasets Jobs alerts

New Evaluation Datasets Jobs, daily.