AI Quality Assurance Engineer, Test Specialist

Job overview

The AI Quality Assurance Engineer, Test Specialist role is remote; compensation is $60–$90 per hour, with a stated commitment of 35 hours per week. Mercor connects creative and technical talent with leading AI research labs. The engineer designs and reviews tests to identify task failures, ambiguity, and gaps in reference solutions. The role uses Python debugging and quality feedback to improve evaluation rigor and identify shortcuts or grading gaps in AI agent runs.

What you'll do

• Design test cases, including edge cases
• review tasks and reference solutions for ambiguity and gaps
• debug with Python
• develop quality checklists and provide actionable feedback to authors
• identify shortcuts and grading gaps in AI agent runs to protect benchmark scores.

What we're looking for

Skills & competencies

ai quality assurance engineer

test specialist

remote

mercor

$60–$90/hour

35 hours/week

python

git

test engineering

quality assurance

ai evaluation

benchmark testing

Work arrangement

• Weekend coverage required

Why this role

Requires an MSc or PhD in a STEM field or equivalent practical experience, at least one year in a relevant role, and proficiency in Python and Git; AI training or model evaluation experience is preferred.

About the employer

Mercor is hiring for this role. Industry: All Other Professional, Scientific, and Technical Services. Sector: 54.