AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability)

Capital One’s Intelligent Foundations and Experiences (IFX) team builds and deploys proprietary AI platforms that help teams across the company enhance products with responsible, scalable machine learning. In this role, you will design and deliver AI systems focused on benchmarking, evaluation, explainability, and the foundations needed to run large language model applications in production.

As an AI Engineer 4, you will work across the full lifecycle of AI software components, from model training through LLM inference, agents and guardrails, and ongoing monitoring. You’ll also help shape the long-term technical vision for foundational AI systems, with an emphasis on reliability, maintainability, and ethical alignment.

What you’ll do

• Collaborate with engineers, research scientists, technical program managers, and product managers to deliver AI-powered products for associates and customers.
• Design, develop, test, deploy, and support AI components including foundation model training, large language model inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Use a broad stack of open source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, and PyTorch, along with cloud services including AWS, Google Cloud, and Azure.
• Introduce state-of-the-art optimization techniques to improve performance and production efficiency, focusing on scalability, cost, latency, and throughput.
• Own end-to-end architecture for complex AI systems, ensuring maintainability, observability, and ethical alignment.
• Define and maintain service-level objectives (SLOs) for AI reliability, including latency, uptime, and model performance drift.
• Partner with infrastructure engineering to optimize GPU/TPU utilization and accelerate inference pipelines.
• Lead cross-functional technical reviews for new AI system deployments, supporting security, data governance, and compliance standards.
• Mentor Principal and Senior Associates on scalable design, performance tuning, and research-to-production translation.

What we’re looking for

• Education: Bachelor’s Degree or Master’s degree in Computer Science/AI or related fields.
• Experience: At least 4 years developing AI and ML algorithms/technologies, or a Master’s degree with at least 2 years of such experience.
• Programming: At least 4 years programming with Python, Go, Scala, CUDA, or Java.

Tools and technologies

• AWS Ultraclusters, Huggingface, VectorDBs, PyTorch
• AWS, Google Cloud, Azure
• Python, Go, Scala, CUDA, Java, C++, C#, Golang
• GPU, TPU

Benefits

• A comprehensive, competitive, and inclusive set of health, financial, and other benefits to support total well-being.
• Performance-based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI).