← Saturday, October 17, 2026AgentsBenchmark

StarSkirmish gives models one hour to code StarCraft bots; GPT-6 Astra leads

starskirmish.com 6 min read Editor score 7/10

The benchmark gives each model one hour in its first-party CLI harness to write a Protoss bot, then scores five runs against human-written and demo bots; GPT-6 Astra leads the reported results. It tests whether agents can use practice games to improve generated code.

What: StarSkirmish is a benchmark in which each LLM gets one hour to write a Protoss bot for StarCraft: Brood War.
How: Bots are evaluated against competitive human-written bots; traces show practice against the strongest roster bots was the strongest score predictor.
Why it matters: The benchmark tests long-horizon reasoning and agentic coding, while its traces expose how models spend time practicing and improving.

The benchmark has an LLM write a bot that is then evaluated against human-written bots. Sketched from the article's own text.

Read on starskirmish.com → Hacker News · 90 comments

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.