ML Software Engineer, Data Plane

Job overview

The ML Software Engineer, Data Plane role is an onsite position based in Cupertino, CA. Compensation is not specified in the posting, and this role supports the Annapurna Labs team in designing inference data planes. This role exists to build software that makes large models run efficiently on custom hardware. The engineer will cover the full inference path, including model execution, memory management, and data movement.

What you'll do

• Develop and optimize compute kernels for custom ML accelerator architecture.
• Implement and validate LLM architectures end-to-end.
• Integrate custom accelerator backends into open-source ML serving frameworks.
• Build and maintain test infrastructure for model correctness.
• Profile and optimize inference workloads to drive latency and throughput improvements.

What we're looking for

Skills & competencies

full-time

cupertino ca

software engineering

machine learning

data plane

inference

llm

c++

optimization

onsite

no on-call

no travel

Work arrangement

• Weekend coverage required

Benefits & perks

• Health insurance, 401(k) matching, paid time off, parental leave, sign-on payments, and RSUs.

Why this role

Focus on inference data plane software for large-scale models.

About the employer

Annapurna Labs (U.S.) Inc. is hiring for this role. Industry: Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services. Sector: 51.