Position Details

Type Full-Time

Experience senior

Exp. Years 5+ years

Education Not specified

Category Software Engineering

About this role

Senior software development engineer on Amazon Stores Foundational AI (SFAI) team, building foundational LLM capabilities that fuse general knowledge with Amazon's e-commerce domain knowledge to power conversational, personalized shopping experiences.

Key Responsibilities

Design and optimize high-performance kernels and low-level acceleration techniques
Drive memory management, parallel computing, kernel fusion, attention optimization, and matrix multiplication efficiency
Partner with scientists, engineering teams and product managers to define requirements, support experimentation, and deliver production-ready systems
Move quickly in ambiguous environments and deliver incrementally across technologies from distributed data processing to ML infrastructure
Develop tooling to accelerate experimentation and generate insights across model quality, latency, throughput, and efficiency metrics

Technical Overview

Hands-on development of high-performance kernels and ML infrastructure for LLM training/inference, focusing on memory management, kernel fusion, attention optimization, and throughput. Collaborates with scientists and product teams to deliver production-ready AI systems using CUDA, TensorRT, PyTorch, and related tools.

Ideal Candidate

The ideal candidate is a senior software engineer with 5+ years of backend software development experience, deep expertise in ML/LLM systems, and hands-on work with CUDA kernels and ML frameworks. They should have mentoring experience and a proven track record of designing scalable, production-ready systems for high-throughput AI workloads.

Must-Have Skills

5+ years of non-internship professional software development experienceExperience with at least one software programming language5+ years of leading design or architecture (design patternsreliability and scaling) of new and existing systemsExperience as a mentortech lead or leading an engineering teamExperience with ML technologiesReinforcement LearningDeep LearningComputer VisionNatural Language Processing (NLP) or related applications

Nice-to-Have Skills

5+ years of full software development life cycleincluding coding standardscode reviewssource control managementbuild processestestingand operations experienceBachelor's degree in computer science or equivalentExperience with Machine Learning and Large Language Model fundamentalsincluding architecturetraining/inference lifecyclesand optimization of model executionor experience in computer architectureExperience with CUDA kernels or ML/low-level kernelsExperience with vLLMSGLangTensorRT or similar platforms in production environmentsor experience working with PyTorch or JAX software

Tools & Platforms

SGLangCUDATensorRTPyTorchJAXvLLM

Required Skills

5+ years of non-internship software development experience5+ years leading design or architecturementoring/tech lead experienceML/NLP/DL/CV/RL experienceCUDA kernelsML frameworks (PyTorch/JAX)TensorRTvLLMSGLangLLMskernel optimizationmemory managementdistributed data processinginference optimization

Hard Skills

C++PythonCUDA kernelsTensorRTPyTorchJAXvLLMSGLangLarge Language ModelsNLPReinforcement LearningDeep LearningKernel optimizationMemory managementAttention optimizationMatrix multiplication optimizationML infrastructureObservability toolingDistributed data processingInference optimization

Soft Skills

LeadershipMentoringTeam collaborationProblem-solvingCommunicationAdaptabilityDecision making

Industry & Role

Industry E-commerce

Job Function Develop and optimize production-grade generative AI systems for Amazon Stores at scale

Role Subtype Software Engineer

Tech Domains Python, Java, Kubernetes, Docker, Linux, Windows Server, JavaScript, SQL / PostgreSQL, Active Directory

Keywords for Your Resume

Sr. Software Dev EngineerSenior Software Development EngineerStores Foundational AILLMLarge Language ModelCUDA kernelsTensorRTvLLMSGLangPyTorchJAXML infrastructureinference optimizationmemory managementkernel optimizationlatencythroughputNLPreinforcement learningdistributed data processing

Deal Breakers

Bachelor's degree not specified or not aligned with CS/Eng, Lack of ML/NLP/DL/CV/RL experience, No demonstrated leadership/mentoring experience

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Sr. Software Dev Engineer, Stores Foundational AI -SFAI

Get matched to jobs like this