← Saturday, October 17, 2026AgentsDeep dive

Harness prompt edits can multiply inference prefill work and energy

neuralwatt.com 13 min read Editor score 8/10

Lab tests found harness edits broke prompt caching; at 92% versus 97% cache rates, uncached input meant about 2.7 times the prefill work, and a cache miss roughly doubled request energy. Harness behavior can materially affect GPU workload and serving costs.

What: The post examines how agent harnesses affect inference energy and token efficiency through prompt-cache behavior.
How: Harness edits to previously sent prompts break exact-prefix caching, forcing the server to recompute from the first difference; cache misses can roughly double request energy.
Why it matters: Keeping prompts stable can reduce prefill work and energy, making harness design an important efficiency lever for large-scale inference.

The harness builds a prompt that the server checks against its KV cache. Sketched from the article's own text.

Read on neuralwatt.com → Hacker News · 24 comments

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.