Webdataset Jobs

1 job found

Today

Foundation Model Data Engineer

Sciforium San Francisco, CA

San Francisco, CA

Today

$155k - $210k/yr Full time Ai Foundation Models Ai/ml Data Engineering Big Data

Apply Now

Job Description

Sciforium is seeking a Foundation Model Data Engineer to own the end-to-end strategy, creation, and curation of massive datasets that power foundation models. This role spans pre-training data, post-training and alignment datasets, and multimodal (vision and video) expansion.

Responsibilities

Lead end-to-end creation of pre-training datasets for LLMs, including defining the data mix across web content, code, books, and technical papers to improve downstream model performance.
Design and implement large-scale pipelines for data cleaning, exact and fuzzy deduplication, and extracting high-quality signals from petabytes of raw, unstructured data.
Develop high-quality post-training datasets, including Supervised Fine-Tuning (SFT) instruction sets, multi-turn dialogues, and preference modeling data for RLHF/DPO.
Drive acquisition and processing of vision and video data, handling multimodal alignment challenges, video compression, and temporal consistency.
Build high-throughput data processing scripts in Python, using multiprocessing and multithreading to support massive ingestion and transformation without bottlenecks.
Perform deep-dive statistical analysis of training corpora to detect biases, knowledge gaps, and quality regressions, maintaining a mathematically balanced “diet” for the model.
Design pipelines to generate high-reasoning synthetic data to augment gaps in natural datasets, using existing models for labeling and refinement.

Requirements

5+ years of industry experience in Data Science or Machine Learning, with a proven track record building and managing datasets for foundation models.
Expert-level skills in high-performance code, including multiprocessing, multithreading, and efficient memory management for data-at-scale workloads.
Experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
Hands-on experience building massive LLM training sets from scratch, including raw web crawls (for example, Common Crawl) and specialized domain data.
Experience creating datasets for RLHF, DPO, and multi-turn instruction following, including human-labeling workflow management and quality gold-set handling.
Familiarity with data-at-scale frameworks and formats such as Spark, Ray, WebDataset, and Parquet.

Technologies

Python
multiprocessing
multithreading
Spark
Ray
WebDataset
Parquet
Common Crawl

Benefits

Medical, dental, and vision insurance
401k plan
Daily lunch, snacks, and beverages
Flexible time off
Competitive salary and equity

Nice-to-Haves

Experience building large-scale image or video datasets from scratch (for example, LAION-style pipelines).
Familiarity with large-scale crawling of multimodal data, including video processing, codecs, and compression challenges.
Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
A Master’s or PhD in a quantitative field with a focus on data-centric AI or information retrieval.

Role Details

Location: San Francisco, CA (onsite)
Salary: USD 155,000 - 210,000 per year
Minimum Experience: 5 years

View full job page →

Get Webdataset Jobs alerts

New Webdataset Jobs, daily.