This role is for one of our clients
Compensation: $70 - $110 per hour
Location: Bay Area, California, United States
Work Arrangement: Hybrid (On-site multiple days per week, as required)
Employment Type: Full-Time, W-2
Work Commitment: 40 hours per week
Initial Engagement: 6 months
Job Overview
We are seeking an experienced, board-certified physician to work directly with the research and program management teams at a leading AI lab, helping advance how next-generation AI models understand, reason about, and respond to real-world clinical challenges.
As a Clinical AI Expert, you will bring your medical expertise and independent clinical judgment to evaluate AI-generated medical content, develop high-quality clinical benchmarks, and establish reference solutions that define what accurate, safe, and evidence-based medical reasoning looks like.
This role is designed for practicing physicians with substantial post-residency experience, strong specialization in a clinical discipline, and the ability to assess medical reasoning beyond surface-level correctness. You will collaborate closely with AI researchers and engineering teams to improve the quality, reliability, and clinical relevance of advanced AI systems.
This is a full-time W-2 employment opportunity with a hybrid work arrangement in the San Francisco Bay Area. You will receive client-issued accounts and equipment and work directly within the client's internal tools and research environment.
Requirements
Key Responsibilities
Clinical Quality Assurance: Review and evaluate clinical knowledge tasks and AI-generated medical responses, identifying gaps in clinical reasoning, missing considerations, unsafe recommendations, and deviations from established medical guidelines.
Instruction Specifications and Reference Solutions: Develop detailed task specifications, create accurate reference answers, and design clinical problems that reflect real-world medical practice and decision-making.
Clinical Benchmark Development: Design challenging, specialty-specific clinical tasks and evaluation datasets to measure the accuracy, depth, safety, and reliability of AI-generated medical reasoning.
Medical Content Evaluation: Assess AI responses across diagnosis, treatment planning, patient management, clinical guidelines, and specialty-specific medical scenarios.
Technical Feedback: Provide clear, structured, and actionable written feedback to help research teams identify weaknesses and improve model performance.
Research Collaboration: Work with AI researchers, engineers, and specialists from adjacent domains to establish consistent evaluation standards and translate clinical expertise into measurable assessment criteria.
Clinical Standards: Help develop medicine-specific evaluation frameworks, tools, and workflows that ensure AI outputs meet professional clinical standards.
Required Qualifications
MD or DO degree from an accredited medical school.
Completed residency training in a recognized medical specialty.
Minimum of 4 years of independent clinical practice following residency. Residency and fellowship training do not count toward this requirement.
Active, unrestricted medical license in at least one US state.
Current board certification in the candidate's respective specialty.
Demonstrated expertise in a defined clinical specialty, including but not limited to:
Internal Medicine
Oncology
Radiology
Emergency Medicine
General or Specialty Surgery
Psychiatry
Cardiology
Neurology
Pediatrics
Other recognized medical specialties and subspecialties
Clear professional progression into senior clinical responsibilities, such as Attending Physician, Medical Director, Division Chief, Associate Professor, Professor, or Chief Medical Officer.
Proven ownership of clinical decisions and experience applying independent medical judgment in patient care.
Practical experience using large language models (LLMs) in a professional setting, with the ability to distinguish clinically sound reasoning from…