← Monday, October 19, 2026AgentsPaper

Interactive exploration raises a VLM’s 3D benchmark accuracy by 22 points

huggingface.co by Yijiang Li et al. Editor score 8/10

ACT3D lets a vision-language model navigate by taking actions and observing rendered images, with reinforcement learning driven mainly by answer correctness; it reports 53.2% average accuracy across six benchmarks, 22 points above its base Qwen model. The approach offers a concrete training recipe for agents that must gather information through interaction.

What

ACT3D lets a vision-language model explore a 3D environment through actions and rendered images before answering a visual question.

Method

After trajectory-supervised fine-tuning, reinforcement learning rewards final-answer correctness without directly supervising exploration choices.

Why it matters

Across six 3D benchmarks, ACT3D improves its base Qwen model's average accuracy by 22 points while using 50–70% fewer input frames.

Read on huggingface.co →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.