← Sunday, September 27, 2026AgentsPaper

WhatWorkedBench measures whether agents can predict experiment effects

huggingface.co by Jingjie Ning et al. Editor score 8/10

The benchmark covers 36 tasks and 1,248 configurations; fitting a Gaussian process to agent observations improves effect recovery from 0.632 to 0.698 in one cohort and from 0.621 to 0.720 in another. It tests whether research agents can learn which experimental changes actually move results.

What: WhatWorkedBench measures agents’ ability to predict how experiment outcomes change when components are changed.
Method: Agents inspect code, choose measurements, and submit a response surface; exhaustive CPU execution supplies reference effects.
Why it matters: It provides a way to assess whether agents can predict experiment effects, a key capability for AI research workflows.

Agents inspect code, select measurements, and submit predicted scores for component configurations. Sketched from the article's own text.

Read on huggingface.co → Code · 2 stars

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.