← Monday, October 5, 2026InferenceBenchmark

Qwen 125B runs locally at up to 94 tokens/s on tested consumer GPUs

github.com 7 min read Editor score 8/10

Strata reports generation speeds of 53–94 tokens/s across quantizations on an RTX 5070 and an RX 9070 XT, with prompt processing also measured. Engineers weighing local inference can compare the speed and hardware trade-offs against server-based deployment.

What

Strata runs the 125-billion-parameter Qwen3.8-Flash-Next model locally on consumer NVIDIA or AMD graphics cards.

How

On two tested gaming PCs, measured answer speeds ranged from 44 to 94 tokens per second, depending on GPU and model size.

Why it matters

Local inference keeps model use on the PC and makes a large model available without relying on a server.

Read on github.com → Hacker News · 333 comments

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.