← Thursday, October 15, 2026AgentsPaper

GroundGUI pairs 586K screenshots with a distillation-to-RL training method

huggingface.co by Qingsong Xie et al. Editor score 8/10

ScreenRef contains 586K screenshots and over 633K referring-expression instances across desktop, web, and mobile; GroundGUI-8B combines supervised fine-tuning with D2RL. The dataset and training recipe target a key grounding bottleneck for GUI agents.

What: GroundGUI introduces ScreenRef, a GUI grounding dataset with screenshots paired with referring expressions and box or point annotations.
Method: ScreenRef covers desktop, web, and mobile; GroundGUI-8B is trained with sequential supervised fine-tuning and D2RL.
Why it matters: The dataset and training method target grounding and generalization challenges engineers face when building GUI agents.

ScreenRef feeds the sequential SFT and D2RL training pipeline for GroundGUI-8B. Sketched from the article's own text.

Read on huggingface.co → Code · 3 stars

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.