Capgemini is hiring an AI Engineer to help build and operate an RLAIF (Reinforcement Learning from AI Feedback) Plane that turns production evidence from customer-facing agents into safer, measurable, and reversible improvements to agent decisions. This onsite contract role in Dallas, TX focuses on governed agent runtime, observability, evaluation harnesses, and GCP deployment, supported by calibrated AI judging and root-cause-driven optimization.
You’ll work on systems where reliability, latency, cost, and incident response matter, with disciplined release controls and a clear separation between optimization paths. The salary range is USD 39 - 61 per hour.
Key responsibilities
- Design and operate production agents and multi-agent workflows using LangGraph, LangChain, and deep-agent patterns, governed, monitored, and versioned from day one.
- Instrument for observability with tracing, telemetry, and failure taxonomies across agents, tools, memory, and knowledge sources so failures are reproducible back to their root cause.
- Own the evaluation harness including golden sets, offline and online evals, and AI-judge pipelines with order-swap bias control and human calibration loops, with reporting focused on business outcomes.
- Optimize using observed failures through one-change-at-a-time experiments, with regression-safe release using hard gates, approval, canary, and rollback.
- Keep optimization paths distinct so harness, contextual-policy, and model-level changes are independently evaluated, including mechanisms to detect capability and knowledge-coverage gaps and propose governed updates.
- Ship and scale on GCP using Vertex AI, GKE/Cloud Run, BigQuery, and Pub/Sub, with CI/CD, IaC, and cost/latency controls.
- Collaborate and lead with architects, data scientists, SRE, and client business stakeholders, documenting decisions and mentoring engineers on agent engineering practice.
Additional responsibility areas include ingesting and analyzing traces, reconstructing sequences of events (tool calls and business flow), creating prompt/context/routing variants, defining evaluation dimensions, building and calibrating AI judges, designing scoring and reward-signal frameworks, and generating autonomous optimization recommendations. Root-cause identification includes hallucinations, context drift, poor prompts, misrouting, policy or compliance issues, safety (harmlessness, bias differences, toxicity), missing or underperforming tools, taxonomy or ontology gaps, and re-configuration opportunities for identifying recommendations and improving project outcomes.
Requirements
- 5+ years of software engineering experience, including 2+ years focused on LLM or agentic applications.
- Demonstrable experience building and running AI agents in production with ownership of reliability, latency, cost, and incident response.
- Hands-on experience with AI observability and evals: tracing frameworks (including LangSmith, Arize Phoenix, and OpenTelemetry), LLM-as-judge design, golden-set curation, and regression testing.
- Deep practical experience with LangGraph and LangChain, including planning, sub-agents, tool use, memory, human-in-the-loop, and state persistence.
- Strong Python engineering, comfortable with async services, APIs, containers, and testing discipline in CI/CD.
- Working understanding of RLAIF/RLHF concepts, prompt and policy optimization, and isolating a single causal change in a complex system.
Technologies
Python, LangGraph, LangChain, LangSmith, Arize Phoenix, OpenTelemetry, Google Cloud, Vertex AI, GKE, Cloud Run, BigQuery, Pub/Sub, CI/CD, IaC, LLM-as-judge
Benefits
- Medical, dental, vision, and retirement benefits
Preferred qualifications
- Cloud experience on Google Cloud (Vertex AI, GKE, Cloud Run, BigQuery, Pub/Sub, IAM); GCP certification is a plus.
- Experience with retrieval, enterprise knowledge bases, and memory systems with provenance, trust scoring, and decay.
- Familiarity with governed release practices in regulated or large-enterprise settings (approval workflows, canary, rollback, audit trails).
- Telecom or large-scale customer-service domain exposure; experience with MLOps tooling and feature flags.
