ServiceNow is hiring a Staff Machine Learning Engineer to join its Agent Orchestration organization in Santa Clara, CA. In this role, you will help shape the core execution harness and runtime capabilities that power enterprise agentic AI, with an emphasis on reliability, observability, evaluation, and the infrastructure needed to run agents safely at scale.
You will own key portions of how agent workflows are routed, executed, measured, and improved in production, while also guiding technical standards across architecture decisions and engineering practices.
What you’ll do
- Own significant parts of the agent execution harness, the infrastructure layer enabling AI agents to reason over real enterprise data and take action across workflows safely at Fortune 500 scale.
- Design and build the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and exposes execution state across multi-step agentic workflows.
- Drive runtime fault tolerance, latency, and throughput for enterprise workflows that cannot fail silently or behave unpredictably.
- Instrument the harness with tracing, cost attribution, and latency visibility to understand production behavior and catch failures before customers are impacted.
- Build prompt management systems including versioning, templating, and systematic evaluation to keep agent behavior stable across model updates and configuration changes.
- Design and own evaluation frameworks such as unit evals, integration evals, and production monitors to measure quality and detect regressions.
- Integrate with and abstract over frontier LLMs, including model routing, fallback strategies, and managing cost and latency trade-offs in production.
- Set technical standards through architecture decisions, code reviews, and coaching, with focus on agentic design patterns and production AI discipline.
- Define where agent logic lives, distinguishing tool calls, sub-agents, hardcoded paths, and human escalation, and establish those standards across the team.
Required qualifications
- 7+ years building production software systems with a track record in reliability, performance, and scalability.
- Hands-on experience shipping generative AI products relied on by production users, beyond prototype or LLM API integration.
- Strong depth in how large language models work, including failure modes, context constraints, and how prompt design shapes behavior at scale.
- Experience with Python, Kubeflow, and SageMaker, Vertex AI, or AzureML, plus SQL.
- Practical prompt engineering experience with designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
- A proven background in eval engineering, including evaluation suites designed, shipped, and used to drive quality decisions in production AI systems.
- System-level cost and efficiency awareness, including reasoning about model routing, inference cost, and latency trade-offs.
- Strong software engineering fundamentals: distributed systems, API design, and testing discipline.
- Comfort operating in fast-moving, ambiguous, startup-like AI product environments.
Technologies
- Python
- Kubeflow
- SageMaker
- Vertex AI
- AzureML
- SQL
- LangChain
- LlamaIndex
Compensation and benefits
- Base pay: USD 176,100 - 308,200 per year
- Plus equity (when applicable), variable/incentive compensation, and benefits
- Health plans, including flexible spending accounts
- 401(k) plan with company match
- ESPP
- Matching donations
- Flexible time away plan
- Family leave programs
Nice to have
- Experience with multi-agent coordination patterns (A2A, MCP)
- Familiarity with agent frameworks such as LangChain, LlamaIndex, or similar
- Prior experience shipping AI systems in enterprise software
- Experience with AI observability tooling (tracing, cost tracking, LLM-specific monitoring)
- Familiarity with cloud-native infrastructure and production troubleshooting (service observability, logging, monitoring, reliability engineering)
Work location: Santa Clara, CA (onsite). ServiceNow may confirm the distance between your primary residence and the closest ServiceNow office using a third-party service to determine eligibility for a work persona.
ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration regardless of race, color, religion, sex, sexual orientation, national origin, age, disability, gender identity, veteran status, or other protected category. Qualified applicants with arrest or conviction records will be considered in accordance with legal requirements.
If you require a reasonable accommodation during the application process, contact globaltalentss@servicenow.com for assistance. For roles requiring access to controlled technology, ServiceNow may need to obtain export control approval and employment is contingent on obtaining any required export license or approval.
