← Monday, September 28, 2026AgentsBenchmark

Adding “Do not guess” cut fabricated extraction fields from 70.7% to 20.2%

earnanhonestdollar.com 4 min read Editor score 7/10

Across 42 page pairs and seven page types, the test measured fabricated fields when extractors were told to return null for missing data. It offers a practical abstention check for agents that extract information for downstream use.

What

- A test measured whether web extractors invent fields missing from pages, using paired pages that differed by one row.

How

- Across 16 models, the “Do not guess” instruction reduced made-up missing fields from 70.7% to 20.2%; every model made up more without it.

Why it matters

- Engineers building agent-based marketplaces can use explicit abstention instructions and checks to assess whether extracted answers are supported by source pages.

Read on earnanhonestdollar.com → Hacker News · 9 comments

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.