Capital One is building foundational AI systems through its Intelligent Foundations and Experiences (IFX) team, with a focus on large language model (LLM) inference and agentic AI. In this role, you will help shape end-to-end architectures that bring research to production, while driving evaluation, governance, observability, and reliability for AI services used by associates and customers.
Role summary
As an AI Engineer 4, you will design, develop, test, deploy, and support AI components across the full lifecycle, including foundation model training, LLM inference, agent and multi-agent workflows, similarity search, guardrails, experimentation, and model evaluation. The position also emphasizes production optimization for scalability, cost, latency, and throughput, alongside maintainability, ethical alignment, and measurable AI reliability.
Responsibilities
- Collaborate with a cross-functional group of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products.
- Build and operate AI software components including foundation model training, large language model inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
- Use a broad stack of open source and SaaS AI technologies, including AWS Ultraclusters, Huggingface, VectorDBs, and PyTorch.
- Develop state-of-the-art foundation model optimization techniques to improve scalability, cost, latency, and throughput for large-scale production AI systems.
- Help define the technical vision and long-term roadmap for foundational AI systems at Capital One.
- Own end-to-end architecture for complex AI systems, focusing on maintainability, observability, and ethical alignment.
- Define and maintain service-level objectives (SLOs) for AI reliability, including latency, uptime, and model performance drift.
- Partner with infrastructure engineering to optimize GPU/TPU utilization and improve inference pipeline performance.
- Lead cross-functional technical reviews for new AI system deployments, ensuring security, data governance, and compliance requirements are addressed.
- Mentor Principal and Senior Associates on scalable design, performance tuning, and research-to-production translation.
Requirements
- Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of experience developing AI/ML algorithms or technologies, or a Master’s degree in a related field plus at least 2 years of such experience.
- At least 4 years of programming experience with Python, Go, Scala, CUDA, or Java.
Technologies
- AWS Ultraclusters, Huggingface, VectorDBs, PyTorch
- Python, Go, Scala, CUDA, Java
- AWS, Google Cloud, Azure
- C++, C#, Golang
Preferred qualifications
- Experience leading AI system development with tradeoffs across cost, latency, throughput, and accuracy.
- 6 years of experience deploying scalable and responsible AI solutions on cloud platforms such as AWS, Google Cloud, Azure, or equivalent private cloud.
- Experience designing, developing, delivering, and supporting AI services.
- Experience developing AI/ML algorithms and technologies such as LLM inference, similarity search and VectorDBs, guardrails, and memory using Python, C++, C#, Java, CUDA, or Golang.
- Experience applying state-of-the-art techniques to optimize training and inference software for hardware utilization, latency, throughput, and cost.
- Experience building agentic AI systems and agentic workflows.
- Passion for staying current with AI research and AI systems and applying novel techniques in production.
- Proficiency designing distributed systems for model training, evaluation, and online inference at petabyte scale.
- Experience defining AI model governance processes, including producibility, lineage tracking, and automated retaining schedules.
- Demonstrated ability to influence architectural decisions across multiple AI product lines or platforms.
Location and compensation
- San Jose, CA (onsite)
- USD 215,200 - 245,600 per year
- Performance-based incentive compensation may be available, which may include cash bonus(es) and/or long-term incentives (LTI).
Capital One also provides a comprehensive, competitive, and inclusive set of health, financial, and other benefits supporting total well-being.
