AI Engineer

Momentic is building an AI-native testing platform, and the team is hiring a hands-on AI Engineer to strengthen the models and agents that power it. This role is centered on improving real-world reliability and efficiency, with work spanning fine-tuning, dataset and evaluation creation, and prompt or context tuning informed by production behavior.

What you’ll do

• Fine-tune models for long-running agents, including agents that determine whether test failures stem from infrastructure, recent code changes, or test authorship.
• Build and curate training and evaluation datasets from production workflows, using recurring failure patterns to identify improvement opportunities.
• Create LLM evaluations and observability that connect offline findings to production behavior, helping the team verify that changes improve outcomes.
• Tune prompts, context, and model behavior to raise task success rates, reliability, and performance while managing latency, token usage, and cost.
• Design repeatable experiments and support publishing benchmarks that demonstrate the core agents’ performance across testing and browser workflows.
• Own technical projects from problem definition through delivery, partnering with the engineering team to make practical tradeoffs and ship improvements.

What you bring

• Relevant software engineering experience, typically 3+ years, including hands-on work on AI-powered capabilities used in production.
• Hands-on experience with supervised fine-tuning, including preparing training data and evaluating model behavior.
• Experience with prompt and context tuning, evaluation datasets, and LLM observability to diagnose failures and measure improvements.
• Strong engineering fundamentals: you can code efficiently and understand both your strengths and areas to grow.
• Practical experience using AI agents to build software by breaking down work, tracking progress, checking results, and redirecting when agents get stuck.
• Comfort with autonomy and ambiguity, including turning unclear problems into plans, managing your work, and delivering outcomes.

Compensation and location

• Location: San Francisco, CA (onsite)
• Salary: USD 175,000 - 250,000 per year

Benefits

• Medical, dental, and vision insurance.
• 401(k).
• Unlimited PTO and sick days, plus company-observed US holidays.
• Daily lunch and a fully stocked kitchen.
• Commuter benefits.
• Team offsites and events.

Nice to have

• Experience with reinforcement learning fine-tuning, RL environments, or LLM-as-judge evaluators.
• Experience with long-running agents, recovery from failed steps, or coordinating multiple agents.
• Experience publishing benchmarks or running rigorous experiments on AI systems.