Mercor is seeking a Senior Machine Learning Engineer focused on Anonymization to build production machine learning systems that score, validate, and improve complex work products under real constraints. This role emphasizes evaluation frameworks, feedback loops, and end-to-end ownership of how models behave in production when correctness is nuanced and labels are imperfect.
Key Responsibilities
- Build machine learning systems that score, validate, and improve complex work products where correctness is nuanced and labels are imperfect.
- Design evaluation frameworks for ambiguous tasks where ground truth may be partial, delayed, or disputed.
- Create feedback loops that convert review, disagreement, correction, and adjudication into measurable improvements to both models and the overall system.
- Own production ML behavior end-to-end, including precision/recall tradeoffs, regression detection, drift, latency, cost, and explainability.
- Improve model quality by selecting appropriate approaches such as prompting, fine-tuning, retrieval, active learning, heuristics, and error analysis.
- Collaborate with backend engineers to integrate inference into durable, long-running workflows while preserving debuggability and human oversight.
- Work quickly on a young, high-ownership codebase where engineering decisions carry long-term architectural impact.
- Operate across models, data, backend systems, and product surfaces, where context switching is expected.
- Debug production ML failures in live, long-running workflows where silent errors matter.
- Partner closely with backend engineers on a stack including Python, Temporal, Postgres, AWS, and LiteLLM.
- Balance automation confidence with human review, including knowing when to defer as well as when to ship.
Required Qualifications
- A track record of shipping ML systems that improved a real product, workflow, or business metric.
- Strong instincts for model quality, evaluation design, error analysis, and production failure modes.
- Comfort operating in ambiguous problem spaces where labels are imperfect and correctness evolves.
- Sound judgment about when to use prompting, fine-tuning, heuristics, retrieval, human review, or simpler product constraints.
- Solid engineering fundamentals across the full ML stack, not limited to modeling.
- Familiarity with LLM applications, model-assisted workflows, evaluation frameworks, or human-in-the-loop ML (strong plus).
- 4+ years of experience.
- A tendency to default to simple, inspectable ML systems that improve quickly and fail in understandable ways.
- Discomfort with shipping model behavior without a clear evaluation story.
- Ability to hold ambiguity without paralysis and make reasonable bets with incomplete information.
- Focus on real-world system output rather than only benchmark performance.
Technologies
- Python
- Temporal
- Postgres
- AWS
- LiteLLM
Compensation and Location
Location: New York, NY (hybrid)
Experience: 4+ years
Salary Range: USD 180,000 - 500,000 per year
Benefits
- Generous equity grant vested over 4 years
- Health, dental, and vision insurance
- 401(k) with company match
- Unlimited paid vacation and sick days
- Parental leave for birthing and non-birthing parents
- Free mental health services
- $200 monthly wellness reimbursement
- $1.5K monthly meal stipend
- $200 monthly laundry reimbursement
- Free Equinox membership
- Up to $15K relocation bonus
What Makes This Role Different
- The architecture is not set; early engineers help define how quality is measured, how models and humans interact, where automation is trusted, and how the system compounds over time.
- The feedback loop is short, with shipped model behavior changes directly and visibly affecting what customers receive.
- The role supports a strategically central product area at Mercor, at a time when frontier AI companies lack good solutions to the problem the team is solving.
About the Role
Frontier AI companies face bottlenecks in expert judgment by needing ways to capture it reliably, validate it at scale, and turn it into durable model behavior. In this position, you will build the machine learning systems powering Mercor’s Frontier Data Products: infrastructure that scores, validates, and improves complex work products where correctness is rarely binary and labels are often noisy, delayed, or disputed. A single job can run for days, interleaving model inference, automated checks, expert review, disagreement resolution, and feedback loops. This role sits at the center of that problem.