About this role
Sr. SDET at Dialpad in Kitchener, responsible for owning test automation and QA frameworks for AI Voice Agent services, building automation tooling, and ensuring high coverage through CI/CD integration.
Key Responsibilities
- Own end-to-end quality for agentic features and workflows
- Design automation tooling for AI/LLM systems
- Develop evaluation frameworks
- Drive 80%+ automation coverage
- CI/CD integration for AI QA
Technical Overview
AI/LLM-driven QA with end-to-end testing across UI, backend, and APIs. Focus on prompt engineering, evaluation strategies, and LLM observability.
Ideal Candidate
The ideal candidate is a senior SDET with extensive experience in AI/LLM-driven testing, strong automation tooling, and the ability to build evaluation frameworks and CI/CD integrations.
Must-Have Skills
own end-to-end quality for agentic features and workflowsdesign and build automation tooling and frameworks for AI/LLM-driven systemsdevelop and maintain evaluation frameworks (evals)drive automation coverage 80%+ for critical AI workflowsintegrate AI quality checks into CI/CD pipelinesbuild tooling for LLM observability and debuggingpartner with Applied AI teams on prompt engineeringmodel selectionand evaluation strategies
Required Skills
test automationquality frameworksautomation toolingend-to-end testingUI testingAPI testingbackend testingprompt flowsagent orchestrationLLm observabilityprompt tracingevaluation frameworksevalsCI/CD pipelinesprompt engineering
Hard Skills
test automationquality frameworksautomation toolingend-to-end testingUI testingAPI testingbackend testingprompt flowsagent orchestrationtool integrationsLLM-driven systemsevaluation frameworksevalsLLM observabilityprompt tracingresponse analysisCI/CD pipelinesprompt engineering
Soft Skills
communicationcollaborationproblem-solvingattention to detailteamwork
Keywords for Your Resume
senior sdetsdettest automationquality frameworksautomation toolingend-to-end testingUI testingAPI testingbackend testingprompt flowsagent orchestrationLLM observabilityprompt tracingevaluation frameworksevalsCI/CD pipelinesprompt engineeringaidialpadqa automationllm observabilityci/cdai testing
Deal Breakers
no experience with AI/LLM test automation, no CI/CD experience, no onsite ability in Canada
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile