← Tuesday, October 6, 2026Models & trainingBenchmark

Qwen3.8 27B answers 167 of 169 worded addition prompts with reasoning enabled

simonwillison.net Editor score 7/10

On a DGX Spark, a Q4_K_M Qwen3.8-27B run with reasoning enabled got 167 of 169 addition prompts right; the one-shot test used one sample per prompt combination. It’s a useful, if narrow, example of evaluating local model arithmetic and the effect of reasoning.

What

A test measured whether Qwen3.8-27B could add positive integers and express exact sums solely in English words.

How

With reasoning enabled, it answered 167 of 169 paired prompts correctly; without reasoning, numeric accuracy was 23.57% across 5,070 cases.

Why it matters

The results show why model evaluations should test both reasoning settings and task difficulty, including large-number arithmetic.

Read on simonwillison.net →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.