ServiceNow is hiring a Staff AI Engineer to help build and take ownership of key components of an agent orchestration execution harness for agentic conversational AI at enterprise scale. This role sits in the Agent Orchestration team, which is responsible for the execution core that helps agents work reliably, observably, and safely in production workflows.
Based in Santa Clara, CA with onsite expectations, you will focus on reliability and production-grade operations for multi-step agent workflows, including prompt infrastructure, evaluation frameworks, and LLM integration.
What you’ll do
- Design and build the agent execution harness, the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and exposes execution state across multi-step workflows
- Own the runtime’s fault tolerance, latency, and throughput, with architecture choices that prevent silent or unpredictable failures in enterprise use cases
- Instrument the harness with tracing, cost attribution, and latency visibility to help teams understand agent behavior in production and detect failures before they affect customers
- Build prompt management systems including versioning and templating, plus systematic evaluation to keep behavior stable through model and configuration changes
- Design and own evaluation frameworks (unit evals, integration evals, and production monitors) to measure agent quality, catch regressions, and enable data-driven decisions
- Integrate and abstract over frontier LLMs, including model routing, fallback strategies, and managing production cost and latency tradeoffs
- Set technical standards through architecture decisions, code reviews, and coaching, with emphasis on agentic design patterns and production AI discipline
- Define where agent logic lives, including standards for tool calls, sub-agents, hardcoded paths, and human escalation
What you bring
- 7+ years building production software systems with demonstrated reliability, performance, and scalability
- Hands-on experience shipping generative AI products, including ownership of AI-powered features used by production customers
- Deep knowledge of large language models, including failure modes, context constraints, and how prompt design affects behavior at scale
- Practical prompt engineering experience: designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles
- A proven track record in eval engineering, with evaluation suites designed, shipped, and used to drive quality decisions in production AI systems
- Cost and efficiency awareness at the system level, including reasoning about model routing, inference cost, and latency tradeoffs
- Strong software fundamentals, including distributed systems, API design, and testing discipline
- Comfort working in fast-moving, ambiguous AI product environments
- Experience with multi-agent coordination patterns such as A2A and MCP
- Familiarity with agent frameworks including LangChain or LlamaIndex (or similar)
- Prior experience shipping AI systems in enterprise software
- Experience with AI observability tooling including tracing, cost tracking, and LLM-specific monitoring
- Familiarity with cloud-native infrastructure and production troubleshooting, including logging, monitoring, and reliability engineering
Technologies
- LangChain
- LlamaIndex
- A2A
- MCP
Compensation and benefits
- Base pay of $176,100 - $308,200 per year
- Equity (when applicable)
- Variable/incentive compensation and benefits
- On Target Earnings (OTE) incentive compensation structure for sales positions
- Health plans, including flexible spending accounts
- 401(k) Plan with company match
- ESPP
- Matching donations
- Flexible time away plan
- Family leave programs
ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration without regard to protected characteristics, and applicants with arrest or conviction records will be considered in accordance with legal requirements.
If you need a reasonable accommodation to complete any part of the application process, or cannot use the online application and need an alternative method to apply, contact globaltalents@servicenow.com.
For positions requiring access to controlled technology under export control regulations, ServiceNow may need export control approval for certain individuals, and employment is contingent on ServiceNow obtaining any required export license or other approval.
Work persona note: ServiceNow assigns work personas (flexible, remote, or required in office) depending on role and location. To determine eligibility, ServiceNow may confirm the distance between your primary residence and the closest ServiceNow office using a third-party service.
