AI Engineer 5 (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation)

The AI Engineer 5 role focuses on building responsible and reliable GenAI platform services. You will help design, develop, and operate AI software components, including foundation model capabilities, guardrails, evaluation, and agentic workflows, using scalable cloud and open-source tooling.

Location

New York, NY (onsite)

Compensation

USD 250,800 - 286,200 per year

Role Summary

Work on GenAI platform services for agentic AI, guardrails, and evaluation. The scope includes foundation model training, LLM inference, agents and multi-agent workflows, similarity search, experimentation, governance, and observability for production AI systems.

Responsibilities

• Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that influence how associates work and how customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components, including foundation model training, large language model inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Use a broad stack of Open Source and SaaS AI technologies including AWS Ultraclusters, Hugging Face, VectorDBs, PyTorch, and more.
• Introduce state-of-the-art foundation model optimization techniques to improve performance across scalability, cost, latency, and throughput for production AI systems.
• Contribute to the technical vision and long-term roadmap for foundational AI systems at Capital One.
• Design, implement, and optimize multi-model orchestration pipelines by integrating LLMs, vector search, and domain-specific models into unified systems.
• Establish and lead cost-performance governance reviews across AI systems, including tracking GPU utilization, model throughput, and inference cost efficiency.
• Lead team design councils or design review boards to maintain technical consistency and compliance with AI engineering standards.
• Mentor Principal and Manager-level AI engineers, supporting cross-domain learning and raising organizational technical maturity.

Required Qualifications

• Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 6 years of experience developing AI and ML algorithms or technologies, or a Master’s degree plus at least 4 years of experience developing AI and ML algorithms or technologies.
• At least 6 years of experience programming with Python, Go, Scala, CUDA, or Java.

Technologies

• AWS Ultraclusters, Huggingface, VectorDBs, PyTorch, AWS, Google Cloud, Azure
• Python, Go, Scala, CUDA, Java, C++, C#, Golang
• Rule-based, retrieval-augmented, generative components

Preferred Qualifications

• Experience leading development AI systems with tradeoffs around cost, latency, throughput, and accuracy.
• 7+ years of experience deploying scalable and responsible AI solutions on cloud platforms (AWS, Google Cloud, Azure, or equivalent private cloud).
• Experience designing, developing, delivering, and supporting complex AI systems.
• Experience developing AI and ML algorithms or technologies (including LLM inference, similarity search, VectorDBs, guardrails, and memory) using Python, C++, C#, Java, CUDA, or Golang.
• Experience applying state-of-the-art techniques for optimizing training and inference software to improve hardware utilization, latency, throughput, and cost.
• Experience building agentic AI systems and agentic workflows.
• Passion for staying current with AI research and applying novel techniques judiciously in production.
• Excellent communication and presentation skills, including the ability to articulate complex AI concepts to peers.
• Experience architecting and integrating heterogeneous AI systems, including rule-based, retrieval-augmented, and generative components, into unified production pipelines.
• Experience defining and enforcing standards for ethical AI deployment, including explainability, fairness, and human-in-the-loop review processes.
• Ability to balance model performance and operational cost through dynamic inference strategies and model compression.
• Experience right-sizing models, instance counts, and hardware types based on requirements such as context length and token inputs/outputs.

Benefits

• Health, financial, and other benefits that support your total well-being.
• Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI).
• A comprehensive, competitive, and inclusive set of health, financial and other benefits.