Apple’s Cloud AI Platform team is looking for a Senior / Staff Machine Learning Engineer in Seattle, WA (onsite) to help advance large-scale model training. In this role, you will shape technical direction across teams while building training recipes, improving stability and performance at scale, and delivering scalable infrastructure for reinforcement learning and post-training.
You will collaborate with researchers and engineers to translate model requirements into practical training features and optimizations that other teams can reuse, from distillation and compression to long-context, mixture-of-experts, and multimodal training. The work also includes deep debugging of training behavior and tight integration with Apple’s data and governance stack to meet privacy and security standards.
Key Responsibilities
- Partner with researchers and ML engineers to develop advanced ML models and convert those needs into training features and performance improvements usable across the platform.
- Develop model distillation and compression approaches, including teacher-student distillation and quantization-aware training, to enable smaller models to retain large-model quality.
- Train models with long-context extension, mixture-of-experts architectures, and multimodal inputs, addressing stability and efficiency challenges for each setup.
- Investigate large-scale training behavior issues such as loss spikes, divergence, and throughput drops, tracing root causes to data, numerics, code, or hardware.
- Build and own scalable, efficient training frameworks and services on GPUs and TPUs using PyTorch or JAX and higher-level frameworks.
- Create reinforcement learning infrastructure for post-training, including rollout and inference services, reward pipelines, and secure sandboxes for model code execution and tool use.
- Integrate the training platform with Apple’s data and governance stack so every training run satisfies privacy and security standards.
- Set technical direction across teams, mentor engineers, and raise the engineering bar.
Minimum Qualifications
- 8+ years of experience in large-scale model training, including reinforcement learning, from research to production.
- 5+ years of experience building and operating large-scale model training systems in production.
- Experience with fine-tuning and distillation techniques such as supervised fine-tuning, parameter-efficient fine-tuning with LoRA, and teacher-student distillation.
- Hands-on experience with at least one higher-level training framework: Megatron-LM, DeepSpeed, TorchTitan, NeMo, AXLearn, or MaxText.
- Deep expertise in PyTorch or JAX, including distributed training features.
- Strong programming skills in Python or C++ and strong written and verbal communication, including a record of leading technical work across teams.
- BS, MS, or PhD degree in Computer Science, Computer Engineering, or equivalent practical experience.
Preferred Qualifications
- Experience with low-precision training and the numerical issues it introduces.
- Experience training large models using data, tensor, pipeline, or expert parallelism across hundreds to thousands of GPUs or TPUs.
- Strong understanding of GPU and TPU architecture, including memory hierarchy, matrix units, and high-bandwidth memory, and how these affect training performance.
- Experience with collective communication libraries such as NCCL and high-speed interconnects such as NVLink, InfiniBand, and TPU ICI.
- Experience profiling and optimizing training workloads with Nsight Systems, PyTorch Profiler, or XProf.
- Experience building reinforcement learning or post-training systems for large models.
- Open source contributions to training frameworks, or publications at top ML venues such as NeurIPS, ICML, or ICLR.
Technologies
- PyTorch, JAX
- LoRA, teacher-student distillation
- Quantization-aware training, supervised fine-tuning
- Megatron-LM, DeepSpeed, TorchTitan, NeMo, AXLearn, MaxText
- NCCL, NVLink, InfiniBand, TPU ICI
- Nsight Systems, PyTorch Profiler, XProf
- Python, C++
Salary & Benefits
Base pay range for this role is USD 175,000 - 308,500 per year, depending on skills, qualifications, experience, and location. Apple notes that base pay is part of a total compensation package, with additional opportunities through discretionary stock programs and other compensation elements as applicable.
- Comprehensive medical and dental coverage
- Retirement benefits
- Discounted products and free services
- Reimbursement for certain educational expenses, including tuition for formal education related to advancing your career
- Participation in Apple’s discretionary employee stock programs, including discretionary restricted stock unit awards
- Employee Stock Purchase Plan, enabling purchase of Apple stock at a discount if voluntarily participating
- Discretionary bonuses or commission payments (as applicable)
- Relocation (as applicable)
- Note: benefit and compensation programs are subject to eligibility requirements and other terms of applicable plans
