DataJobs.io
← Back to all jobs

Job Description

Equinix is building a GenAI platform that needs dependable evaluation and trustworthy agent behavior in real production environments. In this hybrid role in Redwood City, the Principal AI Engineer for Trust & Evaluation sets the technical direction for evaluation and trust capabilities and leads a hands-on team delivering the platform features agent builders rely on.

What you’ll do

  • Own the architecture and roadmap for platform evaluation features, then design and build them, including dataset creation and versioning, golden datasets, synthetic test data, and reusable metric templates.
  • Design and implement the agent scoring approach, including code-based checks, LLM-as-judge, tool-call accuracy, and grounding checks, and determine where each method applies.
  • Create human-review capabilities that compare judge scores against human reviewers, including the agreement level required before teams rely on a judge.
  • Define how evaluations run at scale, including before release, for every change to a prompt, model, or tool, and for agents in production.
  • Set the evaluation standard and release playbooks so agent builders can evaluate agents without needing specialized support.
  • Ensure evaluation results are traceable to the agent, model, prompt, and dataset versions tested, including who approved each release for audit readiness.
  • Design and lead the trust architecture and its build: agent identity, per-user access to connected systems, tool permissions, guardrails, and data boundaries.
  • Lead adversarial testing and red-teaming activities such as prompt injection, out-of-scope requests, and data-boundary tests, and convert findings into fixes and regression tests.
  • Partner with security and governance leaders to translate policy requirements into automated checks.
  • Lead end-to-end investigations of the hardest agent failures and performance problems (retrieval, tool call, prompt, model, data, or infrastructure) until causes are resolved.
  • Establish test and performance tooling, including end-to-end agent tests, load and latency tests, and failure diagnostics.
  • Convert production failures and user feedback into regression tests so recurring defects do not return.
  • Write tested, reviewed, and documented code, contribute to shared libraries and tooling, and review team designs and implementations.
  • Ensure the team builds and tests against security, data-privacy, and responsible-AI requirements.

Required qualifications

  • 7+ years of software, ML, or platform engineering experience, including 3+ years building ML or LLM systems in production.
  • Bachelor’s degree in computer science, engineering, data science, mathematics, or a related field, or equivalent experience.
  • Expert Python skills with strong engineering habits including version control, testing, and code review, plus experience designing backend services or platform features used by many teams.
  • Proven experience designing and leading evaluation, testing, or quality platforms for ML or LLM applications at scale.
  • Deep knowledge of LLM applications and agents, including prompting, retrieval, and tool calling, along with how they fail and how to trace hard failures to root cause using logs and traces.
  • Deep knowledge of evaluation statistics, including sampling, confidence intervals, variance across runs, and measuring reviewer agreement.
  • Ability to write SQL and analyze results.
  • Experience leading technical work across teams and influencing without direct authority.
  • Clear written and verbal communication with senior and non-technical stakeholders.

Tools and technologies

  • Python
  • SQL
  • OAuth
  • Token exchange

Compensation and location

  • Location: Redwood City, CA (hybrid)
  • Targeted pay range (base): USD 246,000 - 370,000 per year
  • Employment: Full time

Benefits

  • Employee Assistance program
  • Health, life, disability and voluntary plans (US benefits)
  • Retirement plan contributions (US benefits)
  • Paid Time Off (PTO) and Paid Holidays (US benefits)
  • Healthcare coverage designed to complement the provincial healthcare system (Canada Core Benefits)
  • Equinix-sponsored retirement or savings plans: Defined Contribution Pension Plan (DCPP), Group Retirement Savings Plan (RRSP), and Tax-Free Savings Plan (TSFA) (Canada Core Benefits)
  • Vacation and Paid Holidays (Canada Core Benefits)

How success will be measured

  • The evaluation and trust roadmap is delivered on schedule with tests and documentation.
  • Teams can evaluate their agents on the platform without specialist help.
  • Every agent release includes an evaluation record with criteria, results, open defects, and an approver.
  • Judge scores agree with human reviewers, and teams use the outputs for release decisions.
  • Trust controls are enforced by the platform, covered by automated tests, and pass security and governance reviews.
  • Defects identified during evaluation become regression tests so the same defect does not return.
  • Engineers on the team grow in scope and skill, with the team’s standards adopted across Equinix.

Similar Jobs