Applied AI Engineer — Conversational Agents (Senior/Staff)

Revin is building conversational AI agents with a substantial emphasis on voice, and is hiring a Senior/Staff Applied AI Engineer (hybrid in New York, NY). In this role, you’ll improve real agent behavior in production by turning customer conversation failures into measurable, experimental improvements, then validating reliability, cost, and quality after rollout. You will also help set the evaluation and experimentation direction at Staff level.

Compensation: USD 225,000 - 325,000 per year.

What you’ll do

• Analyze production conversations by listening to calls, inspecting traces, and collaborating with customer-facing teams to identify recurring failure patterns and prioritize fixes.
• Define quality dimensions and build evaluation datasets, then combine human judgment, automated graders, and production metrics to ensure evaluations reflect what customers actually experience.
• Design reproducible experiments, compare approaches, and use offline evaluations plus controlled production rollouts to understand what improves outcomes, where it works, and what tradeoffs are introduced.
• Investigate turn-taking, interruptions, response latency, transcription errors, speech quality, and spoken naturalness by using both audio signals and system telemetry to diagnose root causes and validate changes.
• Implement improvements, monitor impact in production, and take responsibility for reliability, cost, and regressions alongside quality gains.
• Convert production failures and experimental results into reusable assets such as datasets, evaluations, debugging tools, and release checks to help the team iterate faster.
• At Staff level, shape evaluation and experimentation strategy across agent capabilities, identify highest-impact opportunities, and elevate the team’s ability to diagnose and improve agent behavior while staying hands-on.

What you bring

• A demonstrated track record improving substantial AI agent systems deployed to external customers, including explaining system scale and complexity, failures found, hypotheses, experiments, personal implementation, and proof of improved production outcomes.
• Strong experimental and analytical judgment, including defining baselines, designing informative comparisons, reasoning through noisy results, and learning from experiments that do not confirm the hypothesis.
• Software engineering strength, with the ability to analyze conversational data using code and SQL, debug complex agent behavior, and convert improvements into maintainable production software.
• Practical understanding of agent systems, including how models, instructions, context, tools, and orchestration interact, and balancing quality against latency, reliability, and cost.
• Ownership and communication skills to make progress through ambiguity, share findings and tradeoffs clearly, and partner with engineering and product to deliver improvements to customers.
• A desire to work in an early-stage startup, with energy for broad responsibility, rapid iteration, and high standards, including using AI programming tools heavily and effectively.

Preferred voice experience

• Hands-on experience improving production voice agents is strongly preferred.
• Experience diagnosing conversational timing, speech recognition and generation, audio quality, and end-to-end latency across any provider or stack.
• Open to engineers with exceptional experience improving other customer-facing agents who can demonstrate strong judgment on voice issues and want to deepen voice expertise.

Nice to have

• Experience building simulation, replay, or evaluation tools for multi-turn agents.
• Experience developing human review workflows or calibrating automated evaluations against human judgment.
• Experience at a strong startup or on a small team with substantial ownership.

Benefits

• Meaningful early equity
• Excellent health coverage: $0 employee premiums for individual medical coverage on most plans, plus dental and vision insurance
• 401(k) matching up to 4% of salary (Contribute 5% to receive the full match)
• Unlimited PTO
• Commuter benefits
• Free lunches and dinners in the office
• $100/month gym membership stipend
• Tech equipment of your choice