Position Details

Salary $269K – $307K USD / year

Type Not Specified

Experience lead

Exp. Years Not specified

Education Not specified

Category AI & Machine Learning

About this role

Build and deploy responsible Generative AI-powered conversational experiences at Capital One. You will design, develop, and support AI software components including foundation model training, LLM inference, similarity search, guardrails, evaluation, and observability, while contributing to technical vision and roadmap.

Key Responsibilities

Architect and launch conversational experiences powered by Generative AI
Partner cross-functionally to deliver AI-powered products
Design, develop, test, deploy, and support AI components including foundation model training and LLM inference
Leverage AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, and PyTorch
Develop LLM optimization techniques and contribute to AI systems roadmap

Technical Overview

Role focuses on production LLM systems: foundation model training and large language model inference, similarity search, guardrails, evaluation, experimentation, governance, and observability. Stack highlights AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, and PyTorch, plus LLM optimization for scalability, cost, latency, and throughput.

Ideal Candidate

The ideal candidate is a senior/lead AI engineer who has built and deployed production conversational systems using Generative AI. They have hands-on experience with foundation model training, large language model inference, guardrails, and rigorous model evaluation, plus strong platform experience with AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, and PyTorch.

Must-Have Skills

architect and launch conversational experiencesdesigndeveloptestdeployand support AI software components including foundation model traininglarge language model inferencesimilarity searchguardrailsmodel evaluationexperimentationgovernanceobservabilityleverage AWS Ultraclustersleverage Huggingfaceleverage VectorDBsleverage Nemo Guardrailsleverage PyTorchinvent LLM optimization techniques

Tools & Platforms

AWS UltraclustersHuggingfaceVectorDBsNemo GuardrailsPyTorch

Required Skills

Generative AIconversational experienceslarge language model inferencefoundation model trainingsimilarity searchguardrailsmodel evaluationexperimentationgovernanceobservabilityAWS UltraclustersHuggingfaceVectorDBsNemo GuardrailsPyTorchLLM optimization techniquesscalabilitycostlatencythroughput

Hard Skills

Generative AILarge language model inferencefoundation model trainingsimilarity searchguardrailsmodel evaluationexperimentationgovernanceobservabilityOpen Source AI technologiesSaaS AI technologiesAWS UltraclustersHuggingfaceVectorDBsNemo GuardrailsPyTorchLLM optimization techniquesscalabilitycostlatencythroughputlarge scale production AI systemsconversational experiencesstate of art Generative AI capabilities

Soft Skills

architecting and launching AI systemscross-functional collaborationsharing passion to do the right thingquality focustechnical vision and roadmap contributionpartnership with engineersresearch scientiststechnical program managersand product managersinvent and introduce improvements

Industry & Role

Industry Banking

Job Function Architect and engineer production Generative AI systems for conversational customer experiences.

Role Subtype LLM Engineer

Tech Domains Amazon Web Services, Python, AI & Machine Learning, LLM

Keywords for Your Resume

Distinguished AI EngineerGenerative AIconversational experienceslarge language model inferencefoundation model trainingsimilarity searchguardrailsmodel evaluationexperimentationgovernanceobservabilityAWS UltraclustersHuggingfaceVectorDBsNemo GuardrailsPyTorchLLM optimization techniquesscalabilitycostlatencythroughputstate of art Generative AI capabilities

Deal Breakers

Hands-on experience with foundation model training and large language model inference, Experience implementing guardrails and conducting model evaluation, Experience using AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, and PyTorch

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile