Distributed Training

Distributed Training Data Jobs

3 jobs found

9d ago

Machine Learning Engineer

Unity Technologies SF Mountain View, CA

Mountain View, CA

9d ago

$104k - $152k/yr Full time Apache Airflow Artificial Intelligence Automation

Apply Now

Job Description

Unity Technologies SF offers a comprehensive benefits package and a collaborative, data-driven culture in Mountain View. This onsite role supports ML at scale on the Offline Infrastructure team, focusing on data pipelines for training data generation and distributed model training. The position carries a salary range of USD 104,100 to 152,000 per year and requires a bachelor’s degree.

Benefits

Comprehensive health, life, and disability insurance
Commute subsidy
Employee stock ownership
Competitive retirement/pension plans
Generous vacation and personal days
Support for new parents through leave and family-care programs
Office snacks and refreshments
Mental Health and Wellbeing programs and support
Employee Resource Groups
Global Employee Assistance Program
Training and development programs
Volunteering and donation matching program

Responsibilities

Build and maintain data pipelines that generate training datasets for machine learning models and experimentation
Contribute to infrastructure that supports distributed training workflows (for example PyTorch and Ray)
Work with workflow orchestration tools (such as Airflow, Flyte, or similar) to support multi-stage ML pipelines
Improve reproducibility and reliability through dataset validation, monitoring, and testing
Partner with ML engineers to support experimentation and model iteration
Help optimize performance and efficiency across data processing and training systems
Contribute to the evolution of our offline ML platform architecture as it scales

Requirements

Bachelor's degree in Computer Science, Machine Learning, Systems, or a related field
Strong foundation in machine learning systems, distributed systems, or large-scale data processing (through research or projects)
Experience with Python and working with data-intensive workloads
Familiarity with ML frameworks (e.g., PyTorch, TensorFlow) and/or distributed systems (e.g., Ray, Spark)
Experience (academic or applied) with data pipelines, model training workflows, or large datasets
Strong problem-solving skills and ability to translate research ideas into practical systems
Interest in building scalable, reliable infrastructure for machine learning

Technologies

Python
PyTorch
TensorFlow
Ray
Spark
Airflow
Flyte

Additional information

Relocation support is not available for this position
Work visa/immigration sponsorship is not available for this position

View full job page →

10d ago

Senior Machine Learning Engineer

Placement Force Remote 🌍 Remote

Remote

10d ago

$160k - $260k/yr Full time Senior Artificial Intelligence Data Pipeline

Apply Now

Job Description

Placement Force seeks a Senior Machine Learning Engineer to own the complete lifecycle of a production large language model, from initial training and fine-tuning to scalable deployment and rigorous evaluation.

Responsibilities

Design and run pretraining, continued pretraining, and fine-tuning workflows, including supervised fine-tuning, alignment methods such as DPO and RLHF, as well as LoRA/QLoRA and full-parameter training, all tied to well-defined, measurable goals.
Own data pipeline decisions that materially affect model quality, covering data curation, deduplication, mixture weighting, and contamination checks against evaluation datasets.
Execute and interpret distributed training across multiple GPUs and nodes using FSDP, DeepSpeed, or Megatron-style parallelism, diagnosing scale-specific issues such as loss spikes, stragglers, and checkpoint corruption.
Make and justify tradeoffs between model size, training cost, and downstream performance.
Deploy trained models to production with quantization, batching strategies, KV-cache management, and serving framework selection (for example vLLM, TensorRT-LLM, TGI), accompanied by explicit latency, throughput, and cost targets.
Design for LLM serving failure modes, including tail latency under load, graceful degradation, prompt injection risk, and safe fallback behavior.
Develop the operational capabilities around production models, including monitoring, alerting, and rollback paths, applying the same rigor as for other critical services.
Create and maintain evaluation harnesses that go beyond published benchmarks, incorporating task-specific evaluation sets aligned to real product use cases.
Establish human evaluation protocols where automated metrics fall short, and determine the appropriate contexts for each approach.
Own regression detection to catch quality drops caused by new checkpoints, prompt template changes, or serving optimizations before they impact users.
Contribute to safety and robustness evaluation, including measures of hallucination, adversarial testing, and behavior under distribution shift, as an integral part of the release process.

Requirements

More than four years of applied ML engineering experience, with at least two years directly working on large language models in production.
Proven production deployment experience shipping models that served live traffic, with demonstrated awareness of latency, cost, and quality tradeoffs.
Strong software engineering fundamentals.
Fluency with the modern LLM tooling landscape, including training frameworks, serving frameworks, and evaluation tooling.
Comfort with ambiguity and the ability to define what constitutes acceptable model behavior in the absence of established benchmarks.

Technologies

FSDP
DeepSpeed
Megatron-style parallelism
vLLM
TensorRT-LLM
TGI

About the Role

This role involves owning the full lifecycle of a production large language model, spanning training and fine-tuning through deployment at scale and rigorous evaluation of what actually ships. It requires fluency across research, engineering, and production, because the most challenging problems emerge at the intersections of these domains.

View full job page →

1mo ago

Machine Learning Engineer, Commerce Ads Ranking

TikTok San Jose, CA

San Jose, CA

1mo ago Closed

$156k - $317k/yr Full time Ad Tech Ai Ml Distributed Training

This position is no longer accepting applications (closed Aug 10, 2026).

Job Description

Based in San Jose, CA (onsite), this TikTok role seeks a Machine Learning Engineer to optimize Commerce Ads Ranking models from recall through fine sorting and to advance large-scale distributed training in privacy-preserving environments, with a salary range of USD 156,000 – 316,800 per year.

Responsibilities

Improve ads delivery model performance through feature engineering, architecture choices, automated feature crossing, addressing ad cold start, modeling delayed feedback, and applying multi-task and sequence modeling techniques.
Contribute to algorithm and system co-design, including retrieval algorithms, sample mining strategies, and long sequence handling.
Investigate and optimize large-scale distributed training frameworks, focusing on GPU tuning, feature processing, and synchronous training to boost ads delivery efficiency under privacy-preserving constraints.
Advance next generation recommendation approaches using an LLM-based learning paradigm, covering entity understanding, end-to-end generative recommendation, sequence-only recommendation, and a Mixture of Export framework.

Requirements

Strong coding skills in C++ and Python, with a solid foundation in data structures and algorithms.
Robust background in machine learning and familiarity with models such as FM, LR, GBDT, DNN, and Wide & Deep.
Excellent learning ability and collaborative mindset, with strong analytical and problem-solving skills. Publications in KDD, NeurIPS, SIGIR, or experience in data mining or ML competitions are a plus.
Bachelor's degree or higher in computer science or related fields.

Technologies

C++, Python, LLM

Benefits

Medical, dental, and vision insurance from day one
401(k) savings plan with company match
Paid parental leave
Short-term and long-term disability coverage
Life insurance
Wellbeing benefits
10 paid holidays per year
10 paid sick days per year
17 days of Paid Personal Time, prorated at hire with tenure-based accruals
Eligible for discretionary bonuses or incentives
Restricted stock units

Los Angeles County (Unincorporated) Candidates

Occasional unsupervised interaction with internal or external clients and colleagues
Appropriate handling and management of confidential information and access to IT systems
Exercise sound judgment

View full job page →

Distributed Training Data Jobs

Machine Learning Engineer

Senior Machine Learning Engineer

Machine Learning Engineer, Commerce Ads Ranking

Get Distributed Training Data Jobs alerts