Senior Software Engineer
About the role
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Coding Assistant Evaluator
Type: Contract Compensation: $40–$85/hour Location: Remote Commitment: Flexible, task-based Role Responsibilities
- Review long-form transcripts between users and AI coding assistants.
- Analyze the AI's logic, execution, and stated actions in detail.
- Score each transcript using a 10-point rubric across multiple criteria.
- Optionally write brief justifications citing examples from the dialogue.
- Detect mismatches between claims and actions, such as saying "I'll run tests" but not doing so.
- Work independently and asynchronously to complete tasks on your own schedule.
Qualifications Must-Have
- Senior or Staff Engineers with deep code review experience and execution insight.
- QA Engineers with strong verification and consistency-checking habits.
- Technical Writers or Documentation Specialists skilled at comparing instructions vs. implementation.
Preferred
- Backend or Full-Stack Developers comfortable with function calls, APIs, and test workflows.
- DevOps or SRE professionals familiar with tool orchestration and system behavior analysis.
- Proficiency in Python is helpful.
- Familiarity with other languages like JavaScript, TypeScript, Java, C++, Go, Ruby, Rust, or Bash is a plus.
- Comfort with Git workflows, testing frameworks, and debugging tools is valuable.
Compensation & Legal
- Independent service providers classification.
- Payments issued weekly via Stripe Connect.
Application Process (Takes 20–30 mins to complete)
- Submit your resume to begin.
- If selected, you’ll receive rubric documentation and access to the evaluation platform.
- Most applicants hear back within a few business days.
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
- For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity. ,
Senior Software Engineer
About the role
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Coding Assistant Evaluator
Type: Contract Compensation: $40–$85/hour Location: Remote Commitment: Flexible, task-based Role Responsibilities
- Review long-form transcripts between users and AI coding assistants.
- Analyze the AI's logic, execution, and stated actions in detail.
- Score each transcript using a 10-point rubric across multiple criteria.
- Optionally write brief justifications citing examples from the dialogue.
- Detect mismatches between claims and actions, such as saying "I'll run tests" but not doing so.
- Work independently and asynchronously to complete tasks on your own schedule.
Qualifications Must-Have
- Senior or Staff Engineers with deep code review experience and execution insight.
- QA Engineers with strong verification and consistency-checking habits.
- Technical Writers or Documentation Specialists skilled at comparing instructions vs. implementation.
Preferred
- Backend or Full-Stack Developers comfortable with function calls, APIs, and test workflows.
- DevOps or SRE professionals familiar with tool orchestration and system behavior analysis.
- Proficiency in Python is helpful.
- Familiarity with other languages like JavaScript, TypeScript, Java, C++, Go, Ruby, Rust, or Bash is a plus.
- Comfort with Git workflows, testing frameworks, and debugging tools is valuable.
Compensation & Legal
- Independent service providers classification.
- Payments issued weekly via Stripe Connect.
Application Process (Takes 20–30 mins to complete)
- Submit your resume to begin.
- If selected, you’ll receive rubric documentation and access to the evaluation platform.
- Most applicants hear back within a few business days.
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
- For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity. ,