Data Labeling Jobs in Canada

Job summary

The role involves creating high-quality labeled datasets for NLP and computer vision tasks to improve LLM performance. Responsibilities include executing RLHF workflows, performing QA audits, and scoring prompt factuality and safety.

Job details

About The Role Join Rex.zone’s AI Data Operations team to deliver high-quality labeled datasets that improve large language model evaluation and downstream model performance. You will label and review text, images, audio, and multimodal data while following strict annotation guidelines and quality targets. Key Responsibilities Create accurate training labels for NLP and computer vision annotation tasks (text, image, audio, multimodal). Execute RLHF workflows (preference ranking, rationale writing, policy-aligned scoring) for LLM training pipelines. Perform QA evaluation, audit samples, and correct labeling errors to improve training data quality. Apply annotation guidelines compliance; document edge cases and reduce reviewer variance. Run prompt evaluation to score factuality, helpfulness, safety, and instruction adherence. Use taxonomies for content safety labeling (hate, harassment, self-harm, sexual content, violence) with consistent decision rules. Collaborate with project leads on rubrics, calibration sets, and acceptance criteria tied to model performance improvement. Required Qualifications Mid-Senior experience in data labeling, data annotation, QA evaluation, or LLM evaluation. Strong written English for rubric-based judgments and rationale quality. Ability to follow annotation guidelines compliance and maintain consistency. Familiarity with NLP tasks such as named entity recognition and classification. Comfort working remotely in metrics-driven workflows. Preferred Qualifications Exposure to RLHF and large language model evaluation methods. Experience with computer vision annotation (bounding boxes, polygons, segmentation). Experience labeling sensitive content with content safety policies. Compensation Hourly rate: $30–$50 per hour Remote role across Canada How To Apply Apply through Rex.zone and include relevant experience in data labeling, QA evaluation, and LLM/prompt evaluation. Share examples of guideline-driven work (rubrics, calibration, audits) if available.

What you’ll do

The role involves creating high-quality labeled datasets for NLP and computer vision tasks to improve LLM performance. Responsibilities include executing RLHF workflows, performing QA audits, and scoring prompt factuality and safety.

Requirements

Candidates need mid-senior experience in data labeling or LLM evaluation with strong written English skills. Familiarity with NLP tasks and the ability to follow strict annotation guidelines in a remote, metrics-driven environment is required.

Listed skills

• Évaluation · Preferred
• Organization · Preferred
• Training · Preferred
• Attention to detail · Preferred
• Compliance · Preferred
• safety · Preferred
• Audit · Preferred
• Consistent · Preferred
• Labeling · Preferred
• English · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Labeling
• Data Annotation
• QA Evaluation
• LLM Evaluation
• RLHF
• NLP
• Computer Vision
• Prompt Evaluation
• Content Safety Labeling
• Named Entity Recognition
• Classification
• Multimodal Data Annotation

Job areas

• Data & Analytics
• Technology
• Software
• Science & Research

More jobs from Rex.zone