Senior Lead AI Engineer (FM Hosting, LLM Inference)

Senior Lead AI Engineer role at Capital One, focused on FM Hosting and LLM inference within the IFX team, building and optimizing scalable AI systems for real-time banking applications.

Responsibilities

• Collaborate with a cross functional group of engineers, research scientists, technical program managers, and product managers to deliver AI powered products that transform how associates work and how customers interact with Capital One.
• Design, build, test, deploy, and maintain AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Utilize a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Develop and apply state-of-the-art LLM optimization techniques to improve performance metrics such as scalability, cost, latency, and throughput for large scale production AI systems.
• Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One.

Requirements

• A Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 6 years of experience developing AI and ML algorithms or technologies.
• A Master's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of experience developing AI and ML algorithms or technologies.
• At least 6 years of experience programming with Python, Go, Scala, or Java.

Technologies

• AWS Ultraclusters
• Huggingface
• VectorDBs
• Nemo Guardrails
• PyTorch
• Python
• Go / Golang
• Scala
• Java
• C++
• C#

Benefits

• Health benefits
• Financial benefits
• Performance-based incentives (cash bonuses and long-term incentives)

Overview

At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized customer experiences. Our investments in technology infrastructure and world-class talent — along with our deep experience in machine learning — position us to be at the forefront of enterprises leveraging AI. From informing customers about unusual charges to answering their questions in real

Team Description

The Intelligent Foundations and Experiences (IFX) team is at the center of bringing our AI vision to life at Capital One. We collaborate with partners across the company to advance the state of the art in AI science and engineering, building and deploying proprietary solutions that are central to our business and deliver value to millions of customers. Our AI models and platforms empower teams across Capital One to enhance products with the power of AI and drive tangible business outcomes.

The Ideal Candidate

• You love to build systems, take pride in the quality of your work, and share our commitment to doing the right thing.
• You stay current with the latest research and can translate publications into production ready techniques.
• You adapt quickly, bring clarity to large, undefined problems, ask questions, and dig deep to uncover root causes; you articulate findings concisely and propose new ideas even if unproven.
• You are deeply technical with a strong foundation in engineering and mathematics; your expertise in hardware, software, and AI helps you spot optimization opportunities others miss.
• You are a resilient trailblazer who can forge new paths to achieve business goals when the route is unknown.