AI Jobs in Canada

Job summary

Perform RLHF evaluation, preference ranking, prompt evaluation, response grading, and data labeling across NLP and optionally computer vision tasks. Conduct quality audits, apply content safety policies, document edge cases, and identify systematic issues affecting LLM training data and workflows.

Job details

AI Jobs in Canada (Remote, Full-Time) — Open Role Rex.zone supports distributed AI/ML data operations that improve large language model evaluation and training data quality. This role focuses on production-grade workflows such as RLHF, data labeling, prompt evaluation, QA evaluation, and content safety labeling across NLP and (optionally) computer vision. What You Will Do Execute RLHF evaluation, preference ranking, and rubric-based rationales Perform prompt evaluation and response grading (helpfulness/harmlessness/honesty) Complete data labeling tasks including named entity recognition and text classification Run QA evaluation via audits, spot checks, and inter-annotator agreement workflows Apply content safety labeling policies and maintain annotation guidelines compliance Document edge cases and surface systematic failure modes impacting LLM training pipelines Required Qualifications Mid-Senior experience in AI data operations or applied ML support Strong written reasoning and comfort with rubric-based evaluation Familiarity with RLHF concepts and large language model evaluation Experience with labeling taxonomies (classification, NER, evaluation tasks) Comfort using web-based annotation tools, task queues, and spreadsheets How To Apply Apply via Rex.zone and include a brief summary of your RLHF evaluation, data labeling, prompt evaluation, or QA evaluation experience, plus any examples of improving training data quality or clarifying annotation guidelines.

What you’ll do

Perform RLHF evaluation, preference ranking, prompt evaluation, response grading, and data labeling across NLP and optionally computer vision tasks. Conduct quality audits, apply content safety policies, document edge cases, and identify systematic issues affecting LLM training data and workflows.

Requirements

Candidates should have mid-senior experience in AI data operations or applied ML support, strong written reasoning, and familiarity with RLHF and large language model evaluation. They should also know labeling taxonomies and be comfortable using web-based annotation tools, task queues, and spreadsheets.

Listed skills

• Évaluation · Preferred
• Production · Preferred
• Organization · Preferred
• Training · Preferred
• Attention to detail · Preferred
• Compliance · Preferred
• safety · Preferred
• Time · Preferred
• Labeling · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• RLHF Evaluation
• Preference Ranking
• Rubric-Based Evaluation
• Prompt Evaluation
• Response Grading
• Data Labeling
• Named Entity Recognition
• Text Classification
• Quality Assurance Evaluation
• Inter-Annotator Agreement
• Content Safety Labeling
• Annotation Guidelines
• Large Language Model Evaluation
• Written Reasoning

Job areas

• Technology
• Data & Analytics
• Software
• Security & Safety

More jobs from Rex.zone