Senior Software Engineer | $100-$150/hr

Job summary

Design and refine rubrics and evaluation criteria for preference data, and review and label technical data while identifying subtle quality issues. Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.

Job details

Senior Software Engineer – Python / TypeScript | AI Evaluation Experience: 5+ Years Job Type: Contractor Location: Remote - Canada Preferred Start Date: Immediate Role Overview We are looking for a Senior Software Engineer with strong expertise in Python, TypeScript, JavaScript, or Golang to support cutting-edge AI evaluation and preference-data projects. The ideal candidate has experience working with large codebases, data pipelines, infrastructure, preference data, and rubric creation, along with strong technical judgment and communication skills. Key Responsibilities Design, refine, and iterate on rubrics and evaluation criteria for preference data. Review and label technical data while maintaining a high quality bar. Identify subtle quality issues in AI-generated or preference-based datasets. Build and maintain pipelines and infrastructure supporting data generation, collection, and evaluation. Key Requirements 5+ years of professional software engineering experience. Strong proficiency in Python, particularly building or maintaining pipelines and infrastructure, OR strong experience with Typescript, Javascript. Strong written and verbal communication skills. Ability to explain technical decisions clearly and provide actionable feedback. Strong judgment when assessing technical quality and identifying subtle issues. Candidate Preferences Candidates from Tier-1 technology companies or Tier-1 universities are preferred. A valid and updated LinkedIn profile is mandatory and must be included in the resume. Candidates currently working with Microsoft are not eligible for this requirement.

What you’ll do

Design and refine rubrics and evaluation criteria for preference data, and review and label technical data while identifying subtle quality issues. Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.

Requirements

Requires at least five years of professional software engineering experience and strong proficiency in Python, particularly for pipelines and infrastructure, or strong experience with TypeScript or JavaScript. Candidates should communicate technical decisions clearly, provide actionable feedback, and demonstrate sound technical judgment; candidates from Tier-1 technology companies or universities are preferred, and an updated LinkedIn profile must be included in the resume.

Listed skills

• written communication · Preferred
• Go · Preferred
• JavaScript · Preferred
• TypeScript · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• TypeScript
• JavaScript
• Golang
• Software Engineering
• AI Evaluation
• Preference Data
• Rubric Creation
• Data Pipelines
• Infrastructure
• Data Labeling
• Technical Quality Assessment
• Written Communication
• Verbal Communication
• Technical Judgment

Job areas

• Software
• Technology
• Data & Analytics
• Engineering

More jobs from Codefeast