AI Training Jobs in Canada

Job summary

Perform data labeling and evaluation for NLP, LLM, and computer vision tasks to improve model performance. Execute RLHF ranking, conduct QA checks, and apply content safety labeling while documenting edge cases.

Job details

About The Role AI Training Jobs in Canada focus on building and evaluating AI/ML training data for LLM training pipelines. In this full-time remote role, you will label and evaluate text, image, and multimodal data to improve training data quality and model performance. What You Will Do Perform data labeling for NLP, LLM, and computer vision annotation tasks Execute RLHF ranking and pairwise preference judgments Run QA evaluation checks and maintain annotation guidelines compliance Conduct prompt evaluation and response grading for helpfulness, correctness, and safety Apply content safety labeling (e.g., harassment, self-harm, adult content) Document edge cases, write clear rationales, and escalate ambiguous examples Collaborate asynchronously with cross-functional teams while following secure data handling practices Required Qualifications Mid-Senior experience in AI/ML data operations, evaluation, or large-scale annotation programs Strong analytical writing and rubric-driven decision-making Familiarity with LLM training pipelines, dataset QA, and common annotation error sources Experience with NLP (classification, NER) and/or computer vision annotation workflows Ability to meet throughput targets and maintain high agreement rates in remote settings Workflow and Tools Queue-based task routing with SLAs, audit trails, and spot audits Calibration sessions, gold-standard checks, and feedback loops Issue tracking for edge cases and guideline updates Privacy-first, secure remote work practices Compensation Hourly rate: $30–$50/hr (USD)

What you’ll do

Perform data labeling and evaluation for NLP, LLM, and computer vision tasks to improve model performance. Execute RLHF ranking, conduct QA checks, and apply content safety labeling while documenting edge cases.

Requirements

Requires mid-senior level experience in AI/ML data operations or large-scale annotation programs. Candidates must possess strong analytical writing skills and familiarity with LLM training pipelines and annotation workflows.

Listed skills

• Training · Preferred
• Labeling · Preferred
• Évaluation · Preferred
• safety · Preferred
• Analytical · Preferred
• Audit · Preferred
• Compliance · Preferred
• Decision Making · Preferred
• Teams · Preferred
• Time · Preferred
• Tracking · Preferred
• Organization · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Labeling
• NLP
• LLM Training
• Computer Vision
• RLHF
• Annotation
• QA Evaluation
• Prompt Evaluation
• Content Safety Labeling
• Data Operations
• Analytical Writing
• Dataset QA
• NER
• Multimodal Data Evaluation

Job areas

• Data & Analytics
• Technology
• Software

More jobs from Rex.zone