Medical AI Evaluator (Remote)

Job summary

Design evaluation frameworks, clinical scenarios, and assessment methods to test medical AI systems’ clinical reasoning and decision-making. Identify gaps in AI medical knowledge and collaborate with AI researchers to improve model performance.

Job details

Role Title: Resident Medical Specialist (MD/DO) Role Type: Part-time Location: Remote What does day-to-day look like: Work with research teams to evaluate and improve how AI systems handle clinical reasoning. You'll design evaluation methods that test AI performance on real medical problems where clinical expertise makes the difference. 1. Design systematic evaluation frameworks for medical AI systems 2. Create clinical scenarios that test AI reasoning and decision-making capabilities 3. Build assessment methods that capture the nuance of clinical practice 4. Identify gaps in AI medical knowledge and reasoning 5. Collaborate with AI researchers to improve model performance Candidate Requirements: 1. Licensed physician in active clinical practice (any specialty) 2. Experience with clinical decision-making and evidence-based medicine 3. Interest in how AI can support clinical practice 4. Strong analytical and communication skills Why it Matters: Help shape the next generation of medical AI by ensuring these systems can handle the complexity of real clinical practice. Remote work with flexible scheduling around your clinical commitments. Engagement details: 1. Commitment: flexible engagement, remote working, up to 30 hrs/week 2. Duration: 1 month, with potential extensions based on performance and fit What You’ll Do 1. Design systematic evaluation frameworks for medical AI systems 2. Create clinical scenarios that test AI reasoning and decision-making capabilities 3. Build assessment methods that capture the nuance of clinical practice 4. Identify gaps in AI medical knowledge and reasoning 5. Collaborate with AI researchers to improve model performance Apply Now!

What you’ll do

Design evaluation frameworks, clinical scenarios, and assessment methods to test medical AI systems’ clinical reasoning and decision-making. Identify gaps in AI medical knowledge and collaborate with AI researchers to improve model performance.

Requirements

Applicants must be licensed physicians in active clinical practice, with any specialty accepted. They should have experience in clinical decision-making and evidence-based medicine, an interest in AI’s role in clinical practice, and strong analytical and communication skills.

Listed skills

• Communication Skills · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Clinical Decision-Making
• Evidence-Based Medicine
• Clinical Reasoning
• AI Evaluation
• Evaluation Framework Design
• Clinical Scenario Development
• Assessment Method Design
• Analytical Skills
• Communication Skills
• AI Research Collaboration

Job areas

• Healthcare
• Technology
• Science & Research
• Data & Analytics

More jobs from Joblet-AI