Member of Technical Staff, Evaluations

Job overview

The Member of Technical Staff, Evaluations role is listed as onsite in San Francisco, CA; San Francisco is preferred, and exceptional candidates in other locations may be considered. Compensation is $200,000–$350,000 per year. Handshake AI works with frontier labs on data, evaluation, and post-training challenges. The role develops ways to measure and improve frontier AI systems, partnering with researchers, domain experts, and customers. Its work spans evaluation frameworks, benchmarks, feedback systems, human-data quality, and rapid research and engineering iteration.

What you'll do

• Design evaluation frameworks, benchmarks, and methodologies for frontier language models, agents, multimodal models, and reinforcement-learning environments
• develop reward models, verifiers, graders, and feedback systems
• research evaluation reliability and resistance to shortcutting
• build human-data systems and quality signals
• prototype, evaluate, diagnose failures, and contribute publicly through benchmarks, open-source tools, research, and technical writing.

What we're looking for

Skills & competencies

ai evaluations

frontier ai

research and engineering

llm benchmarks

ai agents

reward models

python

open source

san francisco ca

handshake

$200k–$350k

Work arrangement

• Weekend coverage required

Benefits & perks

• Equity
• 401(k) match
• paid parental leave
• fertility benefits
• medical, dental, and vision
• mental health support
• $500 wellness stipend
• $2,000 learning stipend
• flexible PTO
• 15 holidays and 2 flex days
• commuting support, free lunch, and gym access at the San Francisco office.

Why this role

Early team members can influence technical direction, operating culture, and open-source products; PhD or equivalent research experience sought; strong Python and software-building skills required.

About the employer

Handshake is hiring for this role. Industry: Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services. Sector: 51.