Senior QA Specialist – AI Testing

Job summary

Test large-scale applications and AI-enabled systems using manual and automated approaches across functional, integration, regression, performance, and user acceptance testing. Assess AI response accuracy, hallucinations, prompts, safety, and security across web, mobile, cloud, and on-premises solutions.

Job details

Job Title: Senior Systems Testing / QA Specialist – AI Testing Employment Type: Contract Duration: June 2027 Location: Toronto Ideal Rate: $70- $75 (Based on Experience) Hybrid: 3 days onsite Must haves: Minimum 10+ years of experience in software quality assurance and systems testing for large-scale applications. Minimum 3 years of hands-on experience in AI-assisted testing, including AI-powered QA agents, agentic AI, LLM technologies, or chatbot applications used to create and execute test cases. Demonstrated experience testing AI-enabled systems, including response accuracy, hallucinations, prompts, AI safety, and AI security. Strong experience in test automation and manual testing across functional, integration, regression, performance, and UAT activities. Experience testing web-based applications, mobile platforms, cloud integrations, and on-premises solutions. Hands-on experience with AI testing tools such as DeepEval, Promptfoo, Applitools, SaolaAI, Reflect AI, Copilot Studio Kit, or equivalent solutions. Experience with automation tools such as Selenium, Playwright, Tosca, Micro Focus/OpenText UFT, or Cucumber. Experience with performance testing tools such as JMeter or OpenText LoadRunner. Strong understanding of SDLC, Agile/Scrum, Waterfall, Hybrid, and DevOps methodologies. Note: AI-enabled tools may be used to sort applications based on job-related criteria. All AI generated results are vetted by our team and the decision of which candidates move forward is always made by a human.

What you’ll do

Test large-scale applications and AI-enabled systems using manual and automated approaches across functional, integration, regression, performance, and user acceptance testing. Assess AI response accuracy, hallucinations, prompts, safety, and security across web, mobile, cloud, and on-premises solutions.

Requirements

Requires at least 10 years of software quality assurance and systems testing experience, including at least three years of hands-on AI-assisted testing with AI agents, agentic AI, LLMs, or chatbots. Candidates should have experience with AI testing, automation, and performance tools, and understand SDLC, Agile/Scrum, Waterfall, Hybrid, and DevOps methodologies.

Listed skills

• Accuracy · Preferred
• Integration · Preferred
• safety · Preferred
• Agile · Preferred
• Assurance · Preferred
• Quality assurance · Preferred
• Scrum · Preferred
• Software · Preferred
• Integrations · Preferred
• Organization · Preferred
• Attention to detail · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Software Quality Assurance
• Systems Testing
• AI-Assisted Testing
• AI Testing
• Test Automation
• Manual Testing
• Functional Testing
• Integration Testing
• Regression Testing
• Performance Testing
• User Acceptance Testing
• Web Application Testing
• Mobile Testing
• Cloud Integrations
• AI Safety and Security
• Agile/Scrum

Job areas

• Technology
• Software
• Engineering

More jobs from Bevertec