← Wednesday, October 7, 2026Models & trainingPaper

LoGRA cuts LLM RL training memory by up to 45.7%

huggingface.co by Shaokun Zhang et al. Editor score 9/10

LoGRA stores learning signals as low-rank gradient sketches and uses predicted-KL step control; it reports 45.7% lower average memory and stable 27B-model training for over 1,100 steps on one eight-GPU node. This could make RL post-training feasible where dense Adam runs out of memory.

What

LoGRA reduces RL post-training memory by retaining useful learning signals in low-rank gradient sketches.

Method

The sketches support model updates and policy synchronization; predicted-KL step control estimates policy changes and adjusts update magnitude.

Why it matters

LoGRA cut average training memory by up to 45.7% and enabled stable training of a 27B-parameter model for over 1,100 steps on one eight-GPU node.

Read on huggingface.co →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.