← Sunday, October 11, 2026ResearchPaper

Frontier models struggle to reproduce human ML innovation

epoch.ai 62 min read Editor score 7/10

InnovationEval tests whether models can independently devise ML techniques comparable to human work; the excerpt says recent models made little progress despite thousands of dollars in GPU experiments. The evaluation targets a key gap in claims about automating AI research.

What

InnovationEval tests whether AI can independently discover an ML technique matching a recent human-developed innovation.

Method

The tested frontier models made little progress, despite running experiments using thousands of dollars’ worth of GPU time.

Why it matters

For AI teams, the evaluation targets the full research loop—ideas, implementation, experiments and analysis—rather than isolated coding tasks.

Read on epoch.ai → Hacker News · 76 comments

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.