Wednesday, October 21, 2026

Models & training

8 models & training stories from the Oct 21 edition.

PulseOne beat per story, in page order. Taller beats scored higher.

Progress-aligned recurrence reaches 95.5% accuracy at 16 times training lengthScore 8/10 SWIFT uses a frozen weak denoiser to improve diffusion training without inference overheadScore 8/10 Shopify’s post-trained GraphQL model is 7× faster and about 90% cheaperScore 8/10 50,000 fabricated statistics drew citations from ChatGPT and other sourcesScore 7/10 Gemini enterprise search brings managed RAG to air-gapped Google Distributed CloudScore 7/10 llm-echo 0.5 accounts for tool calls and reasoning tokens in simulated usageScore 6/10 Penmentor uses AI to mark up handwritten math and offer hintsScore 6/10 Google’s SAVE method separates research from storytelling in generative mediaScore 6/10

Paper

Progress-aligned recurrence reaches 95.5% accuracy at 16 times training length

PAR aligns each recurrent iteration with the state after the same number of operations and normalizes recurrent states; a Qwen2.5-0.5B model reaches 95.5% accuracy at 512 moves after training on at most 32. The ablations identify which training choices enable length extrapolation.

WhatProgress-Aligned Recurrence trains each iteration to predict the state after the corresponding number of operations and anchors recurrent-state norms to the first iteration.
MethodOn swap-S_5, a model trained through 32 moves reaches 95.5% full-state accuracy at 512 moves; removing norm anchoring cuts extrapolation below 2.2×.
Why it mattersPer-iteration supervision and norm control offer a way to extend recurrent models to sequential problems deeper than those seen in training.

Original · huggingface.co by Pan Wang et al.36 upvotes

Notes & discussion

Paper

SWIFT uses a frozen weak denoiser to improve diffusion training without inference overhead

SWIFT extrapolates training targets away from a frozen weak denoiser’s predictions, then discards the guide; experiments report improvements across model scales and resolutions. The method targets training quality without adding inference cost or changing the sampler.

Original · huggingface.co by Yanjin He et al.23 upvotes

Notes & discussion

Case study

Shopify’s post-trained GraphQL model is 7× faster and about 90% cheaper

For a context-heavy GraphQL subagent task, Shopify says its post-trained Nemotron model scores slightly above a frontier model, responds 7× faster, and costs about 90% less to serve. The results illustrate how specialized open models can cut latency and inference cost in an agent workflow.

Original · shopify.engineering 5 min read

Notes & discussion

Deep dive

50,000 fabricated statistics drew citations from ChatGPT and other sources

A site of 50,000 fake statistics reportedly attracted more than 10,000 AI sessions and links from over 330 domains, with its numbers appearing in research and investment material. The experiment highlights risks in retrieval, citation and data provenance.

Original · peec.ai 13 min read HN 331 pts · 193 comments

Notes & discussion

Release

Gemini enterprise search brings managed RAG to air-gapped Google Distributed Cloud

The preview offers managed RAG search inside a physical security perimeter, without internet or Google Cloud connectivity, on Google Distributed Cloud with Blackwell Ultra. It addresses the operational burden of building search pipelines for disconnected environments.

Original · cloud.google.com by Rohan Grover5 min read

Notes & discussion

Release

llm-echo 0.5 accounts for tool calls and reasoning tokens in simulated usage

The llm-echo debug plugin now includes tool calls, system prompts, and reasoning tokens in its simulated usage counts. That makes it more useful for testing LLM integrations without making real model calls.

Original · simonwillison.net

Notes & discussion

Tool

Penmentor uses AI to mark up handwritten math and offer hints

Penmentor sends handwritten work from Xournal++ to Claude, Gemini or GPT for annotated feedback, hints and optional animations. It is a useful example of multimodal AI in a focused learning workflow, though the excerpt provides no evaluation.

Original · github.com HN 60 pts · 7 comments

Notes & discussion

Guide

Google’s SAVE method separates research from storytelling in generative media

The method splits factual extraction into a structured brief before a separate storytelling step, rather than asking one prompt to do both. This is a practical way to reduce fabrication in multi-stage generative workflows.

Original · cloud.google.com by Chaithanya Bogadi4 min read

Notes & discussion