Machine Learning Engineer II (Model Training)
Job Description
The Bison Group LLC is seeking a Machine Learning Engineer II to own the model training lifecycle for AI analysis tools. The role focuses on hands-on training, evaluation, and deployment of open-weight language and retrieval models, with additional support for speech-related model adaptation in challenging real-world audio conditions.
Responsibilities
- Train and fine-tune open-weight language models using supervised fine-tuning (SFT) and parameter-efficient methods such as LoRA and QLoRA.
- Increase performance on domain terminology, task instructions, and structured outputs.
- Build instruction and preference datasets from transcripts, surveys, and reference materials.
- Clean, deduplicate, and label data, including generating and validating synthetic examples with subject matter input.
- Train or fine-tune embedding and reranker models to support retrieval-augmented generation (RAG).
- Evaluate retrieval quality for specialized terminology and acronyms.
- Adapt speech recognition, speaker diarization, and voice activity detection models for noisy audio, varied accents, and multi-speaker conversations.
- Build offline evaluation suites and regression tests covering citation and attribution accuracy, hallucination rate, and latency.
- Compare model improvements against established baselines prior to quantization, export, and on-premises serving on GPUs and constrained hardware.
- Maintain tradeoffs across model quality, memory use, inference speed, and throughput.
- Create reproducible training workflows with experiment tracking, versioned datasets, checkpoints, and model cards.
- Package models and dependencies for deployment in environments without internet access.
- Work with product, UX, analysts, and subject matter experts to define useful model behaviors.
- Document experiments and decisions so teammates can retrain, evaluate, and maintain models.
Requirements
- 3+ years of hands-on neural network training or fine-tuning.
- Ability to explain a project end to end, including data preparation, training decisions, evaluation outcomes, and deployment.
- Hands-on fine-tuning of a 7B+ parameter open-weight language model using SFT, LoRA/QLoRA, or full fine-tuning, using PyTorch and Hugging Face Transformers/PEFT, NeMo, or equivalent tools.
- Dataset construction skills: cleaning, deduplication, formatting, labeling, and judgment about training suitability.
- Experience designing task-specific metrics and test sets, and diagnosing overfitting, catastrophic forgetting, and data leakage.
- GPU training fundamentals including mixed precision, gradient accumulation, checkpointing, memory budgeting, and multi-GPU basics.
- Strong Python and Linux skills, including command-line work, CUDA environment management, and dependency management for internet-restricted setups.
- U.S. citizenship and eligibility to obtain and maintain a U.S. security clearance.
Technologies
- Python, Linux, CUDA
- PyTorch, Hugging Face Transformers, PEFT, NeMo
- LoRA, QLoRA, supervised fine-tuning (SFT)
- Embedding models, reranker models, RAG
- Speech recognition, speaker diarization, voice activity detection
- Quantize, experiment tracking, model cards
- On-premises GPUs
- Mixed precision, gradient accumulation, checkpointing, multi-GPU basics
Preferred Qualifications
- iOS development, especially on-device machine learning with Core ML or MLX.
- Training small language models or distilling larger models for edge and on-device use.
- Experience with NVIDIA NeMo or model serving with NIM/TensorRT-LLM.
- Preference tuning with DPO, ORPO, or RLHF, or experience with reward modeling.
- Training embedding or reranker models for RAG applications.
- Speech recognition fine-tuning, speaker diarization, or voice activity detection.
- Quantization and inference tooling such as GGUF, llama.cpp, vLLM, AWQ/GPTQ or MLX.
- Experience with ARM64 or other unified-memory platforms.
- Work in defense, intelligence, or air-gapped environments, including an active Secret clearance or higher.
- Familiarity with qualitative research data, including focus groups, surveys, and interviews.