AI Engineer 5 (FM Hosting, LLM Inference)

Build AI systems that power foundation model training and LLM inference while contributing to Capital One’s AI vision through the Intelligent Foundations and Experiences (IFX) team. This onsite role in McLean, VA combines hands-on engineering with the responsibility to deliver reliable, optimized AI capabilities for teams across the company and for millions of customers.

In this position, you will design and deploy AI software components that support orchestration pipelines, evaluation, guardrails, and observability. You will also focus on improving real-world performance across cost, latency, throughput, and scalability for large scale production AI systems.

What you’ll do

• Partner with engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that change how associates work and how customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components spanning foundation model training, LLM inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Use a broad stack of open source and SaaS AI technologies, including AWS Ultraclusters, Huggingface, VectorDBs, PyTorch, and additional AWS tooling.
• Invent and introduce state-of-the-art foundation model optimization techniques to improve scalability, cost efficiency, latency, and throughput.
• Build and optimize multi-model orchestration pipelines that integrate LLMs, vector search, and domain-specific models into unified systems.
• Establish and lead cost-performance governance reviews across AI systems by tracking GPU utilization, model throughput, and inference cost efficiency.
• Lead team design councils or design review boards to maintain technical consistency and alignment with AI engineering standards.
• Mentor Principal and Manager-level AI engineers, strengthening cross-domain learning and raising organizational technical maturity.

Minimum qualifications

• Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 6 years of experience developing AI/ML algorithms or technologies, or a Master’s degree plus at least 4 years.
• At least 6 years of programming experience with Python, Go, Scala, CUDA, or Java.

Helpful technical experience

• Experience leading AI systems with tradeoffs across cost, latency, throughput, and accuracy.
• 7 years of experience deploying scalable and responsible AI solutions on cloud platforms such as AWS, Google Cloud, Azure, or equivalent private cloud environments.
• Experience designing, developing, delivering, and supporting complex AI systems.
• Experience building LLM inference, similarity search/vector databases, guardrails, and memory using Python, C++, C#, Java, CUDA, or Golang.
• Experience applying state-of-the-art techniques for optimizing training and inference software to improve hardware utilization, latency, throughput, and cost.
• Experience developing agentic AI systems and agentic workflows.
• Ability to architect and integrate heterogeneous AI systems (rule-based, retrieval-augmented, and generative) into unified production pipelines.
• Experience defining and enforcing standards for ethical AI deployment, including explainability, fairness, and human-in-the-loop review processes.
• Demonstrated ability to balance model performance with operational cost through dynamic inference strategies and model compression.
• Experience right-sizing models, instance counts, and hardware types based on requirements such as context length and token input/output.

Compensation & location

• Location: McLean, VA (onsite)
• Salary range: USD 229,900 - 262,400 per year

Benefits

• Incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
• Comprehensive, competitive, and inclusive set of health, financial, and other benefits that support total well-being

Team overview

The Intelligent Foundations and Experiences (IFX) team brings Capital One’s AI vision to life. They work with partners across the company to advance the state of the art in science and AI engineering, build and deploy proprietary solutions central to the business, and deliver value to millions of customers. Their AI models and platforms help teams enhance products with the transformative power of AI in responsible and scalable ways.

Technologies you may use: AWS Ultraclusters, Huggingface, VectorDBs, PyTorch, AWS, Google Cloud, Azure, Python, Go, Scala, CUDA, Java, C++, C#, Golang, LLMs, vector search.