Remote Data Labeling Specialist (Canada)

Job summary

The role involves labeling and evaluating text, image, and multimodal data to improve AI/ML training pipelines for LLMs and computer vision systems. Key tasks include RLHF preference labeling, prompt evaluation, and performing QA audits to ensure dataset consistency.

Job details

Remote Data Labeling Jobs in Canada (Full Time) Rex.zone supports AI/ML training pipelines through data labeling, RLHF evaluation, prompt evaluation, and QA checks. You will apply annotation guidelines compliance to improve training data quality for large language models and computer vision systems. About The Role You will label and evaluate text, image, and multimodal data used to train and validate machine learning models. Typical work includes LLM response grading, RLHF preference labeling, prompt evaluation, entity tagging for NLP, bounding boxes and segmentation for computer vision annotation, and content safety labeling. You will follow annotation guidelines, document edge cases, and complete QA evaluation checks to ensure dataset consistency and high training data quality. Key Responsibilities Produce accurate labels for NLP and computer vision tasks Perform RLHF ranking and pairwise preference judgments for LLM training Execute prompt evaluation and rubric-based scoring for model outputs Apply named entity recognition and taxonomy tagging Complete content safety labeling with clear rationales Run QA evaluation workflows including audits, cross-checks, and error analysis Track annotation guidelines compliance and propose guideline improvements Escalate ambiguous cases and contribute to calibration sessions that improve inter-annotator agreement Required Qualifications Professional experience in data labeling, data annotation, QA evaluation, or trust and safety Strong attention to detail and ability to follow annotation guidelines Comfortable working with web-based annotation tools and spreadsheets Ability to explain decisions clearly using rubrics, rationales, and examples Familiarity with NLP concepts such as named entity recognition and text classification Availability for full-time remote work with reliable connectivity Preferred Qualifications Experience with RLHF workflows, LLM evaluation, and prompt evaluation Exposure to computer vision annotation (bounding boxes, polygons, segmentation masks, keypoints) Understanding of training data quality metrics (accuracy, consistency, coverage, bias) Prior work with content safety labeling and policy interpretation Experience collaborating with AI labs, tech startups, BPOs, or annotation vendors Tools and Workflows You may use labeling platforms, internal QA dashboards, and guideline repositories. Workflows can include gold-standard calibration, blind reviews, inter-annotator agreement checks, and structured error analysis aimed at model performance improvement for production AI systems. How To Apply Visit Rex.zone, search for remote data labeling jobs in Canada, review role requirements, and submit your application. Keep examples of prior annotation work, QA experience, and any RLHF or LLM evaluation exposure ready for screening.

What you’ll do

The role involves labeling and evaluating text, image, and multimodal data to improve AI/ML training pipelines for LLMs and computer vision systems. Key tasks include RLHF preference labeling, prompt evaluation, and performing QA audits to ensure dataset consistency.

Requirements

Candidates must have professional experience in data labeling, QA evaluation, or trust and safety with strong attention to detail. Familiarity with NLP concepts and the ability to work full-time remotely with reliable connectivity is required.

Listed skills

• Évaluation · Preferred
• Production · Preferred
• analysis · Preferred
• Training · Preferred
• Attention to detail · Preferred
• Compliance · Preferred
• safety · Preferred
• Machine learning · Preferred
• Accuracy · Preferred
• Time · Preferred
• Reliable · Preferred
• Labeling · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Labeling
• Data Annotation
• RLHF
• Prompt Evaluation
• QA Evaluation
• NLP
• Computer Vision
• Named Entity Recognition
• Content Safety Labeling
• Text Classification
• Bounding Boxes
• Segmentation
• Taxonomy Tagging
• Inter-annotator Agreement
• Rubric-based Scoring

Job areas

• Data & Analytics
• Technology
• Software
• Science & Research
• Security & Safety

More jobs from Rex.zone