← Sunday, October 4, 2026AgentsDeep dive

Claude’s eval tool builds graders but prompts users to choose failures before reviewing data

hamel.dev by Hamel Husain 4 min read Editor score 8/10

Anthropic’s Claude Code plugin offers build_eval and hill-climb commands; a hands-on test found it pushed users to create an eval before inspecting conversation traces. That workflow critique is useful for teams building trustworthy agent evaluations.

What

- Claude Code’s eval tooling includes build_eval and hill-climb commands to build graders and improve applications against them.

How

- In a trial on leasing-assistant traces, Claude prompted users to choose a failure before reviewing the data, and produced an evaluator bundling four failure types.

Why it matters

- Engineers should review traces before prioritizing failures and keep evals scoped so they can inspect what each grader checks.

Read on hamel.dev →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.