✦ Luna Orbit — AI & Machine Learning

Staff Software Engineer - ML Observability

at Datadog

📍 Boston, Massachusetts, USA; New York, New York, USA Hybrid Posted March 11, 2026
Type Full-Time
Experience senior
Exp. Years 8+ years
Education Not specified
Category AI & Machine Learning

A senior AI/ML engineering role focused on developing and scaling AI systems leveraging Large Language Models and Generative AI technologies.

  • Design and implement LLM observability features
  • Develop scalable AI applications
  • Drive product experimentation
  • Collaborate with cross-functional teams
  • Mentor engineers

Involves building scalable backend architectures, working with LLMs and Generative AI, implementing inference and evaluation pipelines, and developing AI observability tools.

The ideal candidate is a senior AI/ML engineer with at least 8 years of experience in developing and deploying LLM and Generative AI applications. They possess deep knowledge of model internals, inference, evaluation, and AI observability tools, with a focus on building scalable, reliable AI systems.

Building and shipping LLM-powered or GenAI applicationsUnderstanding model internalsInference pipelinesEvaluation techniquesPrompt engineering
AI observability toolsModel evaluationBehavior tracingAI system troubleshooting
Large Language ModelsGenerative AIAI observability tools
Distributed systemsScalable backend architecturesLarge Language ModelsGenerative AIModel internalsInference pipelinesEvaluation techniquesPrompt engineeringAI observabilityMonitoring AI systems
Distributed systemsScalable backend architecturesLarge Language ModelsGenerative AIModel internalsInference pipelinesEvaluation techniquesPrompt engineeringAI observabilityMonitoring AI systems
CommunicationRigorInnovationCollaborationProblem-solving
Industry SaaS
Job Function Lead development of AI observability and scalable LLM applications
Staff Software EngineerML ObservabilityLarge Language ModelsGenerative AIAI systemsInference pipelinesModel evaluationPrompt engineeringAI observabilityMonitoring AI systemsDistributed systemsScalable backend architectures

Lack of experience with LLM or Generative AI, No understanding of model internals or inference pipelines, Less than 8 years of relevant experience, Unwilling to work in a hybrid environment

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile