Member of Technical Staff, Evaluation

About the role

Every claim we make to a customer rests on an evaluation: a held-out set of real tasks, a scoring method, and an honest accounting of what changed. You will own that discipline across the company.

You will work with customers' domain experts to turn fuzzy quality standards into gradeable tasks, build and maintain our evaluation harness, and make sure our training signal is not quietly leaking into our test sets.

What you'll do

• Design task suites and rubrics for long-horizon, multi-step agent workflows
• Build LLM-as-judge and programmatic graders, and measure how well they agree with humans
• Maintain strict separation between training and evaluation data
• Produce the before/after reports customers use to decide whether to deploy
• Publish internal and, where appropriate, external write-ups of what we learn

Requirements

• Strong statistical intuition and the habit of asking whether a result is real
• Solid Python and experience building data-heavy tooling
• Experience evaluating LLM systems beyond standard benchmarks
• Excellent writing; your reports are the product

Compensation & benefits

• Base salary: $180,000 to $290,000
• Meaningful early-stage equity
• Medical, dental, and vision coverage
• 401(k)
• Relocation support to San Francisco
• Daily lunch in the office and a learning stipend

About Clad Labs

Clad Labs is an applied AI lab building systems to train, evaluate, and continuously improve production agents. We turn the data a company already generates, real requests, tool calls, and human corrections, into training signal, and we measure every change against held-out tasks before it ships.

We are a small team working in person in San Francisco. We care about doing careful work on hard problems, and about being honest with customers about what improved, what regressed, and what we do not know yet.

Clad Labs is an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.