Business Domain Expert, AI Model Evaluation

Job overview

The Business Domain Expert, AI Model Evaluation role is remote. Compensation is $70–$100 per hour. Mercor connects creative and technical talent with leading AI research labs. The expert completes a domain-specific task, evaluates and ranks model-generated outputs, and provides evidence-based rationales and feedback on the task and study design. The work applies professional judgment in administration, marketing, human resources, or accounting to assess AI model performance.

What you'll do

• Complete a realistic domain-specific task using provided source files
• review five model-generated outputs, rank them from 1 to 5 without ties, score each from 0 to 100, and write evidence-based rationales citing specific output details
• provide structured feedback on the task and study design
• work independently to meet short-turnaround deadlines. The posting prohibits use of large language models or AI assistants at every stage.

What we're looking for

Skills & competencies

business domain expert

ai model evaluation

ai trainer

administration

marketing

human resources

accounting

model output review

evidence-based evaluation

remote

mercor

$70–$100 per hour

Work arrangement

• Weekend coverage required

Why this role

Requires 5+ years of hands-on professional experience in administration, marketing, human resources, or accounting; the posting states that AI assistants are prohibited throughout the work.

About the employer

Mercor is hiring for this role. Industry: All Other Professional, Scientific, and Technical Services. Sector: 54.