Position Details
About this role
Scale AI seeks a Senior AI Infrastructure Engineer to design and operate platforms for scalable, reliable serving of LLMs and internal LLM capability discovery.
Key Responsibilities
- Build and maintain fault-tolerant, high-performance systems for serving LLMs
- Build an internal platform to empower LLM capability discovery
- Collaborate with researchers and engineers to optimize models for production and research use cases
- Conduct architecture and design reviews to uphold best practices
- Develop monitoring and observability solutions
Technical Overview
Backend-focused ML infra stack with languages Python/Go/Rust/C++, containerization (Docker/Kubernetes), cloud infrastructure (AWS/GCP), Terraform; LLM serving concepts like rate limiting, token streaming, load balancing.
Ideal Candidate
The ideal candidate is a senior ai infra engineer with 5+ years of backend systems experience, strong LLM serving knowledge, and proven capability to design scalable cloud-based serving platforms.
Must-Have Skills
Nice-to-Have Skills
Tools & Platforms
Required Skills
Hard Skills
Soft Skills
Industry & Role
Keywords for Your Resume
Deal Breakers
5+ years backend systems experience, LLM serving & routing experience, Onsite in San Francisco or New York
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile