Research Scientist — AI and Language Model Evaluation

Job overview

The Research Scientist role is onsite in San Francisco, CA. Compensation is not specified. The role works with researchers and engineers on evaluation of advanced AI systems and large language models. The scientist develops benchmarks and evaluation methods to assess AI models before enterprise deployment. The work includes evaluating newly released models, building datasets and benchmarks, improving automated text evaluation, and collaborating with engineering, AI labs, and enterprise customers.

What you'll do

• Evaluate newly released AI models
• create benchmarks, including hiring labelers, constructing datasets, and writing white papers
• improve methods for automated evaluation of generated text
• work with engineering to implement and scale evaluation methodologies
• collaborate with AI labs and enterprise customers to understand evaluation needs.

What we're looking for

Skills & competencies

research scientist

san francisco ca

ai evaluation

large language models

benchmarking

evaluation methodologies

python

pytorch

tensorflow

nlp

dataset construction

relocation support

Work arrangement

• Weekend coverage required

Benefits & perks

• Lunch and dinner provided
• relocation support
• health and dental benefits
• 401(k)
• unlimited paid time off.

Why this role

The posting highlights applied AI/ML research, benchmarking, evaluation methodologies, and language models. Listed technologies include Python, PyTorch, TensorFlow, Django, AWS, React, TypeScript, Flask, Git, and CI/CD. Relocation or transportation support is offered.

About the employer

The SeventhHire LLC is hiring for this role. Industry: Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology). Sector: 54.