About the role
AI testing Engineer
Experience: 8+ years Remote (Canada)
Video Testing experience with AWS environment
We are seeking an AI Testing Engineer to validate and assure quality of AI/ML systems powering real-time contextual advertising on live video streams. The role involves testing multi-modal AI outputs (video + text), ensuring accuracy of detected moments, and automating validation workflows. This role is critical for native moments validation and system-level QA in a real-time streaming environment.
Key Responsibilities
Validate AI-generated outputs against labelled datasets and expected outcomes Perform content validation for native moments detection across video streams Design and implement automated testing frameworks (Python-based) Build validation harness for: JSON parsing and comparison Structured output validation using defined schemas Conduct multi-modal validation (video, audio, transcript outputs) Execute system integration and end-to-end testing Track accuracy metrics, defect logging, and root cause analysis Automate regression and validation pipelines across datasets Collaborate with Applied Scientists to validate LLM and GenAI outputs Work within AWS environment using services such as: S3 (data storage).Lambda (processing). CloudWatch (monitoring) Required Skills & Experience: Strong Python scripting for automation and testing. Experience in AI/ML model validation and QA workflows.
Hands-on with: JSON parsing, schema comparison Data validation pipelines
Experience in testing: Multi-modal AI systems (vision + NLP + audio) LLM outputs and generative AI pipelines. Good understanding of: Structured output validation AI accuracy evaluation techniques Familiarity with AWS cloud environment (S3, Lambda, logging/monitoring) Experience with media/video content analysis workflows preferred Ability to handle mixed manual + automated testing approaches
We are an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
Not the right fit? Search for AI testing Engineer jobs in Canada
About Enterprise Solutions Inc.
Enterprise Solutions, Inc. is a technology-intensive services company. We develop and deliver software and engineering solutions to our partners; we work relentlessly to keep our clients happy, and we measure our business success with the trust we earn from them.
Enterprise Solutions offers implementation services for VMS, MSP, ERP, cloud, analytics and digital transformation. We are a certified Minority Business Enterprise (MBE).
We have extensive experience in end-to-end software development using a wide array of technologies. From vendor management to smart automation, ESI will install, implement, and integrate your business infrastructure so that you can operate at full capacity.
Similar Jobs
About the role
AI testing Engineer
Experience: 8+ years Remote (Canada)
Video Testing experience with AWS environment
We are seeking an AI Testing Engineer to validate and assure quality of AI/ML systems powering real-time contextual advertising on live video streams. The role involves testing multi-modal AI outputs (video + text), ensuring accuracy of detected moments, and automating validation workflows. This role is critical for native moments validation and system-level QA in a real-time streaming environment.
Key Responsibilities
Validate AI-generated outputs against labelled datasets and expected outcomes Perform content validation for native moments detection across video streams Design and implement automated testing frameworks (Python-based) Build validation harness for: JSON parsing and comparison Structured output validation using defined schemas Conduct multi-modal validation (video, audio, transcript outputs) Execute system integration and end-to-end testing Track accuracy metrics, defect logging, and root cause analysis Automate regression and validation pipelines across datasets Collaborate with Applied Scientists to validate LLM and GenAI outputs Work within AWS environment using services such as: S3 (data storage).Lambda (processing). CloudWatch (monitoring) Required Skills & Experience: Strong Python scripting for automation and testing. Experience in AI/ML model validation and QA workflows.
Hands-on with: JSON parsing, schema comparison Data validation pipelines
Experience in testing: Multi-modal AI systems (vision + NLP + audio) LLM outputs and generative AI pipelines. Good understanding of: Structured output validation AI accuracy evaluation techniques Familiarity with AWS cloud environment (S3, Lambda, logging/monitoring) Experience with media/video content analysis workflows preferred Ability to handle mixed manual + automated testing approaches
We are an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
Not the right fit? Search for AI testing Engineer jobs in Canada
About Enterprise Solutions Inc.
Enterprise Solutions, Inc. is a technology-intensive services company. We develop and deliver software and engineering solutions to our partners; we work relentlessly to keep our clients happy, and we measure our business success with the trust we earn from them.
Enterprise Solutions offers implementation services for VMS, MSP, ERP, cloud, analytics and digital transformation. We are a certified Minority Business Enterprise (MBE).
We have extensive experience in end-to-end software development using a wide array of technologies. From vendor management to smart automation, ESI will install, implement, and integrate your business infrastructure so that you can operate at full capacity.