← Saturday, October 17, 2026AgentsPaper

FDE-Gym benchmarks agents on ambiguous, end-to-end customer delivery

huggingface.co by Dongxu Yang et al. Editor score 8/10

FDE-Gym contains 95 delivery tasks across 20 customer scenarios and 18 sectors, scoring outcomes against customer success metrics while penalizing interaction costs. Its tasks test requirement discovery and live-system work missing from issue-based agent benchmarks.

What: FDE-Gym benchmarks agents on 95 end-to-end delivery tasks adapted from real-world projects across 18 sectors.
Method: Agents work in simulated customer environments, clarifying requirements with scripted stakeholders and operating modifiable customer systems.
Why it matters: The benchmark highlights a delivery gap: in the realistic setting, the best configuration scored 0.56 on average and resolved 14 tasks without incident.

Agents interact with stakeholders and customer systems to deliver working solutions. Sketched from the article's own text.

Read on huggingface.co → Code · 97 stars

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.