Diffbrief Engineering daily

· No. 18

Tuesday, October 13, 2026

The day in one minute

Inference work is attacking memory limits from two directions: ReplaySSM reduces state traffic, while Janas streams model experts from SSD. Meanwhile, research on hidden neural networks and device-level self-replication shows why capability evaluations need to probe beyond visible behavior. Together, the stories put efficiency and trustworthy evaluation on the same engineering agenda.

  1. ReplaySSM speeds hybrid-model decoding with checkpointed state and input buffers
  2. Researchers hide encrypted neural networks inside host model weights
  3. Tests find current models can complete steps toward self-replication on IoT devices
  4. Janas streams model experts from SSD and reports 24 tok/s on a 48 GB model
Collected
436
Scored
297
Kept
50
Sources
41
Today's pulseOne beat per story, in page order. Taller beats scored higher.

Must read

Inference Deep dive

ReplaySSM speeds hybrid-model decoding with checkpointed state and input buffers

ReplaySSM stores a checkpoint plus recent inputs, cutting state-memory traffic roughly in half and speeding decoding kernels 1.46–1.80x; speculative throughput rises 1.85–2.10x. The design targets memory traffic, rollback, and precision trade-offs in hybrid-model serving.

  • WhatReplaySSM represents SSM state as a checkpoint plus a buffer of recent inputs.
  • HowIt computes outputs from the checkpoint and buffer, writing a new checkpoint when the buffer fills.
  • Why it mattersSeparating output computation from state updates targets state traffic, speculative-decoding rollback, and quantization error accumulation.
read read when full checkpoint buffer outputs
ReplaySSM computes outputs from a checkpoint and input buffer, then updates the checkpoint when the buffer fills. Sketched from the article's own text.

Original · developer.nvidia.com by Michelle Horton15 min read

Notes & discussion

Models & training Paper

Researchers hide encrypted neural networks inside host model weights

The authors describe backdoors that decrypt and run hidden neural networks only after a secret trigger, and report testing elicitation methods including fine-tuning and gradient-based trigger search. The result challenges the assumption that capability evaluations can reliably expose hidden model behaviors.

Original · arxiv.org 3 min read HN 185 pts · 100 comments

Notes & discussion

Tool of the day

Performance & OS Tool

PS2 games run in WebAssembly after ahead-of-time recompilation

The project translates PlayStation 2 machine code to C++, then compiles it to WebAssembly; Gran Turismo 4 reportedly runs at 50 fps in Chrome on an integrated GPU. Its recompilation approach and performance make it a useful case for engineers working on runtimes and compatibility.

  • WhatPS2 Web Recomp runs PlayStation 2 games in a browser by statically recompiling them rather than using an emulator.
  • Howeerecomp translates game machine code into C++, which is compiled to WebAssembly and linked against a PS2 system and hardware reimplementation.
  • Why it mattersPreserving CPU quirks, vector timing, and hardware behavior shows the compatibility work needed to run complex legacy software on a browser stack.

Original · github.com 11 min read HN 289 pts · 148 comments

Notes & discussion

AI Radar Full radar →

Trending models Hugging Face

#ModelLikesMove
1 google/embeddinggemma-2feature extraction 2K+203 –
2 jialinyyzz/humanizertext generation 1.4K+198 –
3 Cloudflare/clefimage text to text 2.1K+61 –
4 ConwayResearch/Underdog-Saluki-27B-1.0text generation 383+45 ▲ 2
5 Qwen/Qwen3.8-27Bimage text to text 18K+65 –

Fast-rising repositories GitHub, created this week

#RepositoryStarsMove
1 storytold/wordcraft 9.8K+2.8K ▲ 1
2 wheresryan22/anatomy 7K+2.8K ▲ 1
3 storytold/gridcraft 4K+829 ▲ 4

Agents

10 stories Topic page →

Paper

MASS optimizes multi-agent workflows through self-supervision

MASS repeatedly optimizes multi-agent workflows and distills their collective output into a self-supervision signal; the authors report improved score per output token over a base model on research benchmarks and provide code. The method and evidence make it relevant to teams building agent systems and evaluating their efficiency.

  • WhatMASS repeatedly optimizes multi-agent workflows and distills collective intelligence into self-supervision for recursive self-improvement.
  • MethodIt frames the model being improved as its own optimizer and evaluator when stronger teachers or reliable human feedback may be unavailable.
  • Why it mattersMASS improves score per output token over the base model on research benchmarks.

Original · pub.sakana.ai 18 min read HN 135 pts · 79 comments

Notes & discussion
distilled into used as multi-agentworkflows collectiveintelligence self-supervisionsignal
MASS turns multi-agent work into collective intelligence used as a self-supervision signal. Sketched from the article's own text.
3 more stories

Inference

5 stories Topic page →

Benchmark

Janas streams model experts from SSD and reports 24 tok/s on a 48 GB model

On a 32 GB laptop, Janas reports 24 tok/s generating with a 48 GB Qwen3-Next-80B model, versus 2.8 tok/s for llama.cpp, and also reports faster prompt processing. The results could inform trade-offs for running models larger than available RAM.

  • WhatJanas-LLM runs large mixture-of-experts models by streaming needed experts from an NVMe SSD while keeping recurring ones in memory.
  • HowIt reports 24 tok/s for a 48.4 GB Qwen3-Next-80B-A3B model on a 32 GB RAM laptop, compared with 2.8 tok/s for llama.cpp.
  • Why it mattersStreaming weights from storage lets models larger than RAM run on ordinary computers, making storage and memory traffic central to inference design.

Original · github.com 14 min read HN 119 pts · 37 comments

Notes & discussion
streams recurring ones NVMe SSD experts memory
Janas streams needed experts from NVMe storage and keeps recurring experts in memory. Sketched from the article's own text.

Models & training

7 stories Topic page →

Paper

SynthSelf uses a same-family aligner to continually post-train small models

SynthSelf outsources synthetic self-distillation to a minimally larger, same-family aligner and combines it with off-policy supervision and rejection sampling; evaluations cover four acquisition datasets and 14 retention benchmarks. The method addresses continual learning for low-latency, smaller models.

  • WhatSynthSelf is a continual post-training method that uses synthetic self-distillation to help small models learn new capabilities.
  • MethodIt uses a minimally larger, same-family aligner for in-distribution rewrites, with off-policy supervision and rejection sampling.
  • Why it mattersAcross 4 acquisition datasets and 14 retention benchmarks, SynthSelf outperformed 6 methods across 3 small-model scales.

Original · huggingface.co by Woosung Koh et al.41 upvotes

Notes & discussion

Architecture

5 stories Topic page →

Case study

Meta uses idle inference capacity for ads recommendations without hurting baseline performance

Meta’s system expands recommendation work onto opportunistic inference capacity using two elasticity levers while protecting baseline performance. It offers a practical example of extracting more value from an inference fleet without compromising its primary workload.

  • WhatMeta’s ads recommendation system aggregates locally idle inference capacity into a shared pool for optional recommendation work.
  • HowRank More Ads (RMA) increases ranking candidates, while Serve Better Models (SBM) routes more requests to a stronger model; isolation and load shedding protect baseline serving.
  • Why it mattersThe elasticity levers let recommendations use changing spare capacity while preserving baseline latency and reliability.

Original · engineering.fb.com 10 min read

Notes & discussion
capacity capacity opportunisticcapacity pool Rank More Ads(RMA) Serve BetterModels (SBM)
Opportunistic capacity supports optional ranking and model-serving work through two elasticity levers. Sketched from the article's own text.

Performance & OS

6 stories Topic page →

Tool

Wi-Fi 7 router streams raw radio samples as a software-defined receiver

The project repurposes a router’s MediaTek radio to capture 14-bit IQ samples at up to 320 Msps and stream them over 10 GbE, using custom firmware. Its radio and streaming pipeline make for a substantial performance-engineering project.

  • WhatW1700K SDR turns a Wi-Fi 7 router into a software-defined receiver that streams raw radio samples to a Linux host.
  • HowThe MT7996 captures IQ samples, the Airoha AN7581 SoC sends them over 10 GbE, and host software provides a shared stream and spectrum display.
  • Why it mattersThe design shows how radio capture, high-throughput transport, and shared host-side consumers fit into a live data pipeline.

Original · github.com 5 min read HN 336 pts · 90 comments

Notes & discussion
radio DMA sample export IQ stream samples MT7996 Airoha AN7581SoC 10 GbE w1700k-stream w1700k-spectrum
Radio samples move from the router across 10 GbE to host streaming and spectrum software. Sketched from the article's own text.

Cloud & infra

2 stories Topic page →

Release

Meta announces liquid-cooled racks, 102.4T switches, and an AI transport protocol

Meta is sharing IcePack liquid-cooled network racks, an IceCube 102.4T switch platform, a GPU scale-out fabric, and the MetaRoCE transport protocol. These infrastructure changes matter to engineers building large AI clusters.

  • WhatMeta announced open networking infrastructure for AI workloads, including a liquid-cooled rack, a 102.4T switch platform, a scale-out fabric, and a transport protocol.
  • HowIceCube switch blades provide 128×800G non-blocking capacity inside IcePack racks, and run FBOSS using the OCP SAI interface.
  • Why it mattersThe announced scale-up, scale-out, and multi-site networking work targets larger AI clusters and gigawatt-scale infrastructure.

Original · engineering.fb.com 11 min read

Notes & discussion

Databases

2 stories Topic page →

Deep dive

Postgres TOAST compression can trigger when a value grows by one byte

The piece shows that a 2,000-byte value remains uncompressed while a 2,001-byte value is compressed to 26 bytes, then explains row-page limits and TOAST behavior. It also covers Postgres 19's shift to LZ4, useful context for storage and compression trade-offs.

  • WhatTOAST compresses variable-length values when a Postgres row gets too large, and can move them out of the row if compression is insufficient.
  • HowA 2,000-character repeated value stays uncompressed, while 2,001 characters triggers LZ4 compression to 26 bytes.
  • Why it mattersKnowing the row-size trigger and compression behavior helps engineers reason about storage costs; Postgres 19 defaults to LZ4 when supported.

Original · planetscale.com by Ahmed Darwich12 min read

Notes & discussion

Security

2 stories Topic page →

Guide

Updated cryptographic guidance says asymmetric choices have changed

The guidance says post-quantum transition plans now target completion by 2029, while symmetric cryptography remains sound and many asymmetric recommendations have changed. Engineers choosing cryptographic defaults should review the updated advice.

  • WhatUpdated cryptographic guidance says the post-quantum transition should finish by 2029 and that asymmetric recommendations have changed since 2024.
  • HowIt recommends, among other choices, X25519+ML-KEM-768 for key exchange and ML-DSA-44 for asymmetric signatures; it says symmetric cryptography remains fine.
  • Why it mattersEngineers choosing security primitives for infrastructure need to account for the changing post-quantum guidance and protocol choices.

Original · words.filippo.io 18 min read HN 91 pts · 30 comments Lobsters 58 pts · 3 comments

Notes & discussion

Vulnerability

Danfoss refrigeration controllers had authentication bypass and command injection flaws

Research on the Danfoss AK-SM 800A found a hidden code-of-the-day mechanism that could bypass authentication and a command-injection flaw leading to remote code execution; Danfoss addressed the issues in firmware. The case illustrates risks in internet-connected operational technology, though it concerns a specific product.

Original · claroty.com 10 min read Lobsters 1 pts

Notes & discussion

Languages & tools

6 stories Topic page →

Deep dive

Zig comptime and arena allocation make type-safe HTML templating practical

The author uses Zig's comptime and arena allocators to build high-level HTML generation without manually freeing the many strings involved. It is a concrete example of applying low-level language features to web development.

  • Whatzt is a Zig templating library designed for type-safe HTML generation with escaped inputs and direct output writing.
  • HowTemplates are parsed and transpiled to Zig during zig build; generated code writes to std.Io.Writer, and the compiler checks template parameters.
  • Why it mattersCompile-time checks and arena allocation help make low-level web development manageable without runtime template errors or manual string cleanup.

Original · lalinsky.com 6 min read HN 243 pts · 93 comments Lobsters 29 pts

Notes & discussion
parsed by transpiles to writes output templates zig build Zig code std.Io.Writer
Templates are transpiled during the build into Zig code that writes rendered output to a writer. Sketched from the article's own text.

Practice & careers

1 story Topic page →

QGIS proposes banning AI-generated contributions

QGIS contributors cite review overload and the risk of unreviewed or insufficiently reviewed code as reasons for a strict ban on AI contributions. The policy debate highlights how AI-generated submissions can shift work onto maintainers.

  • WhatQGIS contributors are discussing a policy that would prohibit AI-generated contributions, replacing an earlier human-in-the-loop approach.
  • HowSupporters say AI-assisted submissions increase review pressure, can leave pull requests unreviewed, and may weaken contributor learning and trust.
  • Why it mattersThe debate highlights how AI-generated code can shift costs onto maintainers and affect review capacity in large open-source projects.

Original · github.com 22 min read HN 373 pts · 402 comments

Notes & discussion

Sources today: Airbnb Engineering, AWS Architecture, AWS Machine Learning, ByteByteGo, Cloudflare Blog, Google DeepMind, DuckDB, Figma Engineering, Google Cloud, GitHub Engineering, Google Research, Hugging Face, Import AI, InfoQ, Interconnects, Latent Space, Lobsters (AI, distributed, performance), LWN.net, Engineering at Meta, Microsoft Research, Metadata (Murat Demirbas), NVIDIA Technical Blog, The Old New Thing, OpenAI, Pinterest Engineering, PlanetScale, Postgres Weekly, The Pragmatic Engineer, PyTorch, Simon Willison, WebKit, GitHub (new repositories), Hugging Face trending models, Hugging Face Daily Papers, Hacker News, Hacker News (AI and systems search), Lobsters, LangGraph releases, Ollama releases, Transformers releases, vLLM releases. Headlines, summaries and diagrams are written by gpt-6-luna from each source's own text; links and images come from the sources themselves. Follow the link before relying on a claim.

RSS · JSON · Markdown · Pulse · Search · Previous days

Your note

Private to you. Stored in this browser, and synced to your account when you are signed in.