STEM Engineering Jobs Canada

Job summary

Design and maintain scalable systems and data pipelines for labeled data, preference collection, evaluation, and quality assurance supporting AI/ML training workflows. Build internal tools and workflow automation, integrate labeling platforms, monitor data quality and system performance, and collaborate with ML, Data Ops, Product, NLP, and computer vision teams.

Job details

Rex.zone is hiring for remote, full-time engineering roles focused on building, testing, and scaling systems that support real-world AI/ML training workflows, including RLHF, data labeling, QA evaluation, prompt evaluation, and LLM training pipelines. About The Role You will design and implement end-to-end engineering solutions connecting dataset ingestion, labeling operations, preference data collection, evaluation pipelines, and quality assurance controls. You will partner with ML, Data Ops, and Product to translate annotation guidelines into scalable systems and ship reliable services that improve training data quality and throughput. Key Responsibilities Build and maintain data pipelines for labeled data, preference data, and evaluation datasets Develop internal tools for RLHF, prompt evaluation, and QA evaluation Implement workflow automation for annotation guidelines compliance, audit trails, and reviewer consensus Integrate data labeling platforms and content safety labeling into ML pipelines Define monitoring for data quality, latency, cost, and model evaluation metrics Collaborate with NLP and computer vision teams to support NER, classification, and CV annotation tasks Harden systems for security, privacy, and access control in distributed remote teams Required Qualifications Mid-senior engineering experience building production systems Strong programming and systems fundamentals with experience designing scalable services Experience with data engineering concepts (ETL/ELT, schemas, versioning, lineage) Familiarity with ML pipelines, dataset management, and evaluation workflows Ability to work cross-functionally with data labeling and QA teams Strong written communication for requirements, runbooks, and incident retrospectives Preferred Qualifications Experience supporting LLM training pipelines, RLHF tooling, and prompt evaluation frameworks Experience building quality systems for annotation guidelines compliance and adjudication Familiarity with MLOps practices, CI/CD, and observability for data-intensive systems Compensation Competitive hourly rate: $30–$50 per hour. How To Apply Apply via Rex.zone and highlight experience with scalable system design, data pipelines, and any RLHF, data labeling platforms, QA evaluation, or LLM training pipeline work.

What you’ll do

Design and maintain scalable systems and data pipelines for labeled data, preference collection, evaluation, and quality assurance supporting AI/ML training workflows. Build internal tools and workflow automation, integrate labeling platforms, monitor data quality and system performance, and collaborate with ML, Data Ops, Product, NLP, and computer vision teams.

Requirements

Requires mid-senior engineering experience building production systems, strong programming and systems fundamentals, and experience with scalable services and data engineering concepts. Candidates should be familiar with ML pipelines, dataset management, and evaluation workflows, and communicate effectively across engineering, labeling, and QA teams; experience with RLHF, LLM training, MLOps, CI/CD, and observability is preferred.

Listed skills

• CI/CD · Preferred
• Quality assurance · Preferred
• Workflow Automation · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Scalable System Design
• Data Pipelines
• ETL/ELT
• Dataset Management
• RLHF Tooling
• Prompt Evaluation
• Quality Assurance
• Data Labeling
• Machine Learning Pipelines
• MLOps
• CI/CD
• Observability
• Workflow Automation
• Security and Access Control
• NLP
• Computer Vision

Job areas

• Technology
• Software
• Engineering
• Data & Analytics

More jobs from Rex.zone