← Monday, September 28, 2026AgentsBenchmark

SWE-Serve finds live-serving checks cut coding-agent pass rates to 45.9%

developer.nvidia.com by Elizabeth Goodman 9 min read Editor score 9/10

Across 19 tasks that start a live server, patches passed 45.9% with full verification versus 69.4% without live-serving checks. The benchmark exposes a significant gap between local tests and production-path validation for inference-engineering agents.

What

SWE-Serve benchmarks coding agents on 53 SGLang inference-engineering tasks derived from 83 merged pull requests.

How

For 19 live-serving tasks, patches passed 45.9% with full checks versus 69.4% without live-serving checks.

Why it matters

It tests whether repository-level changes work through the live serving path, exposing failures other checks can miss.

Read on developer.nvidia.com →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.