Foundation Model Data Engineer

Sciforium is seeking a Foundation Model Data Engineer to own the end-to-end strategy, creation, and curation of massive datasets that power foundation models. This role spans pre-training data, post-training and alignment datasets, and multimodal (vision and video) expansion.

Responsibilities

• Lead end-to-end creation of pre-training datasets for LLMs, including defining the data mix across web content, code, books, and technical papers to improve downstream model performance.
• Design and implement large-scale pipelines for data cleaning, exact and fuzzy deduplication, and extracting high-quality signals from petabytes of raw, unstructured data.
• Develop high-quality post-training datasets, including Supervised Fine-Tuning (SFT) instruction sets, multi-turn dialogues, and preference modeling data for RLHF/DPO.
• Drive acquisition and processing of vision and video data, handling multimodal alignment challenges, video compression, and temporal consistency.
• Build high-throughput data processing scripts in Python, using multiprocessing and multithreading to support massive ingestion and transformation without bottlenecks.
• Perform deep-dive statistical analysis of training corpora to detect biases, knowledge gaps, and quality regressions, maintaining a mathematically balanced “diet” for the model.
• Design pipelines to generate high-reasoning synthetic data to augment gaps in natural datasets, using existing models for labeling and refinement.

Requirements

• 5+ years of industry experience in Data Science or Machine Learning, with a proven track record building and managing datasets for foundation models.
• Expert-level skills in high-performance code, including multiprocessing, multithreading, and efficient memory management for data-at-scale workloads.
• Experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
• Hands-on experience building massive LLM training sets from scratch, including raw web crawls (for example, Common Crawl) and specialized domain data.
• Experience creating datasets for RLHF, DPO, and multi-turn instruction following, including human-labeling workflow management and quality gold-set handling.
• Familiarity with data-at-scale frameworks and formats such as Spark, Ray, WebDataset, and Parquet.

Technologies

• Python
• multiprocessing
• multithreading
• Spark
• Ray
• WebDataset
• Parquet
• Common Crawl

Benefits

• Medical, dental, and vision insurance
• 401k plan
• Daily lunch, snacks, and beverages
• Flexible time off
• Competitive salary and equity

Nice-to-Haves

• Experience building large-scale image or video datasets from scratch (for example, LAION-style pipelines).
• Familiarity with large-scale crawling of multimodal data, including video processing, codecs, and compression challenges.
• Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
• A Master’s or PhD in a quantitative field with a focus on data-centric AI or information retrieval.

Role Details

• Location: San Francisco, CA (onsite)
• Salary: USD 155,000 - 210,000 per year
• Minimum Experience: 5 years