Data Quality Analyst (Remote)

Job summary

Review datasets and task outputs for accuracy, completeness, and compliance with instructions, applying evaluation rubrics consistently and documenting decisions in ambiguous cases. Identify and escalate recurring issues, and provide clear, actionable feedback on findings and potential resolutions.

Job details

Role Title: Data Analyst (AI Evaluation) Role Type: Contractor Location: Remote, US-based. Data Analysts to contribute their knowledge and expertise to a customer’s data quality project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. Scope of Work 1. Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines. 2. Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions. 3. Apply structured rubrics and evaluation criteria consistently across large data sets. 4. Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear. 5. Escalate patterns or recurring data issues for further analysis rather than addressing them individually. 6. Provide clear, actionable written and verbal feedback pinpointing issues and suggesting resolution paths. 7. Maintain meticulous attention to detail while working through high volumes of repetitive data review assignments. Preferred Qualifications 1. 3–5+ years of experience in data review, data quality, quality assurance, or analytical roles. 2. Proven expertise in reviewing, validating, and critiquing data or deliverables based on explicit criteria. 3. Exceptional ability to follow detailed written instructions and apply complex evaluation rubrics precisely. 4. Demonstrated comfort with ambiguity, including making justifiable decisions when guidelines are incomplete. 5. Advanced attention to detail and error-spotting capabilities across repetitive or high-volume work. 6. Strong written and verbal communication skills for documenting findings and providing feedback. 7. Baseline fluency with spreadsheets (sorting, filtering, simple formulas); SQL or hands-on data querying is a plus. 8. Experience with data annotation, labeling, or rubric-based evaluation is advantageous. What You’ll Do 1. Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines. 2. Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions. 3. Apply structured rubrics and evaluation criteria consistently across large data sets. 4. Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear. 5. Escalate patterns or recurring data issues for further analysis rather than addressing them individually. 6. Provide clear, actionable written and verbal feedback pinpointing issues and suggesting resolution paths. Apply Now!

What you’ll do

Review datasets and task outputs for accuracy, completeness, and compliance with instructions, applying evaluation rubrics consistently and documenting decisions in ambiguous cases. Identify and escalate recurring issues, and provide clear, actionable feedback on findings and potential resolutions.

Requirements

The role prefers 3–5 or more years of experience in data review, data quality, quality assurance, or analytical work, with strong rubric application, attention to detail, and communication skills. Spreadsheet fluency is expected; SQL, data querying, and experience with data annotation or rubric-based evaluation are advantageous.

Listed skills

• Data Validation · Preferred
• Quality assurance · Preferred
• Data analysis · Preferred
• Attention to detail · Preferred
• written communication · Preferred
• SQL · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Quality Review
• Data Validation
• Quality Assurance
• Data Analysis
• Rubric-Based Evaluation
• Attention to Detail
• Error Detection
• Instruction Following
• Judgment Under Ambiguity
• Written Communication
• Verbal Communication
• Spreadsheet Skills
• SQL
• Data Annotation
• Issue Escalation

Job areas

• Data & Analytics
• Technology

More jobs from Joblet-AI