Copy of LLM Model Response Evaluation
Evaluate AI-generated content across various modalities including text, images, and video against quality rubrics for factuality and consistency. Conduct independent research on unfamiliar topics to ensure accurate and well-supported evaluation decisions.
- Remote
- Ottawa, Ontario, Canada
- Posted Aug 6, 2026
- Apply by Feb 2, 2027
- 1 position
More jobs you can apply to directly
Similar opportunities posted by employers hiring on Jobs.ca, with no external application form.
Job summary
Our client, a global technology company that helps businesses build, train, and manage AI systems is looking for experts to evaluate model-generated content against defined quality rubrics such as factuality, consistency, aesthetics, and other evaluation criteria. Job Description Evaluating UI widgets, infographics, image factuality, side-by-side comparisons, and similar AI evaluation activities. The work may involve text, images, audio, video, HTML widgets, PDFs, or combinations of these modalities. The work is domain-agnostic and may cover topics across arts, culture, history, science, engineering, and more. Resources will be expected to independently research unfamiliar topics using trusted sources before making evaluation decisions. Each task will include detailed project guidelines within the evaluation platform. Qualifications 3+ years of hands-on experience in LLM / GenAI data evaluation. Bachelor's Degree required Ability to research unfamiliar topics using trusted sources and make well-supported judgments. Comfortable evaluating content across multiple modalities Additional Information Flexible and remote work Variable workload: Accept or decline tasks based on your availability No guaranteed hours: Workload may vary weekly
What you’ll do
Evaluate AI-generated content across various modalities including text, images, and video against quality rubrics for factuality and consistency. Conduct independent research on unfamiliar topics to ensure accurate and well-supported evaluation decisions.
Requirements
Requires a Bachelor's degree and at least 3 years of hands-on experience in LLM or Generative AI data evaluation. Candidates must be comfortable working with multiple content modalities and performing independent research.
Listed skills
- Data analysisPreferred
Other relevant skills
Identified from the job description. Confirm important requirements above.
- LLM Evaluation
- Generative AI Data Evaluation
- Factuality Assessment
- Multimodal Content Evaluation
- Research
- Quality Rubric Application
- Data Analysis
Job areas
- Data & Analytics
- Technology
- Software
- Science & Research
- Consulting
Additional details
- Minimum education
- Bachelor’s degree
- Minimum experience
- 2+ years
- Apply by
- Feb 2, 2027
- Posting language
- English
- Working hours
- 40 hours per week
- Location requirements
- Country, Ottawa, Ontario, Canada
- Seniority
- Not Applicable
- Application method
- Direct apply is available