Evaluation And Metrics Data Jobs

1 job found

Today

Staff Software AI Engineer

Rivian Palo Alto, CA

Palo Alto, CA

Today

Full time Backend Developer Agentic Ai Platform Artificial Intelligence

Apply Now

Job Description

Rivian’s Applied AI team is building internal AI applications powered by large language models and agentic workflows. In this Staff Software AI Engineer role (onsite in Palo Alto, CA), you will define the technical direction for an agent platform that integrates enterprise knowledge and tools, with a focus on reliability, governance, and scalable production delivery.

What you’ll work on

You will establish and own the agent platform architecture used to orchestrate long-running, multi-step agents. This includes persistent state management, tool integration, and permission-aware access to enterprise data. You will also build evaluation infrastructure that gates production releases, helping ensure internal AI applications meet quality and performance expectations over time.

Define and own agent platform architecture for orchestration, persistent state, tool integration, and evaluation-driven production gating
Create reusable patterns, interfaces, and engineering standards so capabilities can be adopted across the organization
Set the bar for reliability, observability, and evaluation of AI systems, ensuring production apps meet targets for quality, latency, and cost
Lead design and implementation of the most complex components, deliver reference implementations, and resolve critical integration and reliability challenges
Evaluate emerging models, frameworks, and tools, leading build-versus-buy and vendor selection decisions and translating insights into a roadmap
Partner with Security, Legal, and Compliance to embed authorization, auditability, and data governance into the platform
Raise the engineering bar through design and code reviews, mentoring, and participation in hiring

Requirements

8+ years of professional software engineering experience, including technical leadership of complex production systems
Experience shipping and operating LLM-based or agentic applications with real users in production, supported by measured quality, adoption, and business impact
Deep knowledge of agent system design: orchestration, tool calling, retrieval and grounding, context and memory management, durable execution, verification, and evaluation of nondeterministic systems
Backend and distributed systems expertise: APIs, asynchronous execution, state management, fault tolerance, and observability at scale
Expert-level Python and experience deploying cloud services with containers and automated delivery pipelines
A record of setting technical direction and engineering practices adopted beyond your team, with demonstrable impact on systems and engineers
Experience integrating with enterprise systems and data under security, privacy, and permission constraints
Ability to translate ambiguous needs into practical technical solutions, with clear communication to engineers, domain experts, and executives
Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience

Preferred qualifications

Experience with multi-agent systems including verification or adversarial checking, persistent state, and recovery or human-approval mechanisms
Experience with multimodal applications, including document or engineering-drawing understanding, vision models, or computer-use agents
Familiarity with product development tooling: CAD, PLM, simulation, CAE, or physics-informed machine learning
Experience improving AI systems through failure analysis, feedback loops, model adaptation, or fine-tuning
Experience building AI infrastructure or capabilities adopted by multiple teams, or forward-deployed experience at an AI lab or agent-infrastructure company

Technologies: Python, containers

View full job page →

Get Evaluation And Metrics Data Jobs alerts

New Evaluation And Metrics Data Jobs, daily.