Position Details

Type Full-Time

Experience senior

Exp. Years 5+ years

Education Not specified

Category QA & Testing

About this role

Sr. SDET at Dialpad Vancouver, responsible for owning test automation and QA frameworks for AI-driven Voice Agent services, building evaluation strategies, and integrating AI QA into CI/CD.

Key Responsibilities

Own end-to-end quality for agentic features and workflows
Design automation tooling and frameworks for AI/LLM systems
Develop and maintain evals
Drive 80%+ automation coverage
Integrate AI QA into CI/CD

Technical Overview

AI/LLM-driven QA with end-to-end testing across UI, backend, and APIs. Focus on prompt engineering, evaluation strategies, and LLM observability.

Ideal Candidate

The ideal candidate is a senior SDET with strong experience in AI/LLM test automation, evaluation frameworks, and CI/CD integration, capable of driving high automation coverage in a Vancouver-based environment.

Must-Have Skills

own end-to-end quality for agentic features and workflowsdesign and build automation tooling and frameworks for AI/LLM-driven systemsdevelop and maintain evaluation frameworks (evals)drive automation coverage 80%+ for critical AI workflowsintegrate AI quality checks into CI/CD pipelinesbuild tooling for LLM observability and debuggingpartner with Applied AI teams on prompt engineeringmodel selectionand evaluation strategies

Required Skills

test automationquality frameworksautomation toolingend-to-end testingUI testingAPI testingbackend testingprompt flowsagent orchestrationLLm observabilityprompt tracingevaluation frameworksevalsCI/CD pipelinesprompt engineering

Hard Skills

test automationquality frameworksautomation toolingend-to-end testingUI testingAPI testingbackend testingprompt flowsagent orchestrationtool integrationsLLM-driven systemsevaluation frameworksevalsLLM observabilityprompt tracingresponse analysisCI/CD pipelinesprompt engineering

Soft Skills

communicationcollaborationproblem-solvingattention to detailteamwork

Industry & Role

Industry SaaS

Job Function Develop and maintain automated test infrastructure for AI-driven Dialpad services in Vancouver, ensuring robust QA and CI/CD integration.

Role Subtype SDET

Keywords for Your Resume

senior sdetsdetqadialpadvancouvercanadaonsitetest automationqa automationaillmprompt engineeringci/cdevaluation frameworksprompt flowsautomation toolingbackend testingapi testingui testingllm observabilityprompt tracingai testing

Deal Breakers

no experience with AI/LLM test automation, no CI/CD experience, unable to work onsite in Vancouver

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile