← Tuesday, October 6, 2026AgentsPaper

OSWorld-Pro evaluates computer-use agents across subgoals and action failures

huggingface.co by Zhilin Wang et al. Editor score 9/10

OSWorld-Pro provides over 300 tasks, 2,800 subgoals, and 67,000 human annotations; its top model scores 75.7% versus 83.4% on OSWorld. Process-level evaluation can expose whether agents fail on inputs, clicks, or other steps rather than only judging final deliverables.

What

OSWorld-Pro evaluates computer-use agents through over 2,800 subgoals across more than 300 tasks.

Method

Human annotations ground LLM judges that score subgoal fulfillment, exposing progress and process-level failures such as irrelevant actions and click mistakes.

Why it matters

Engineers can use its subgoal-level failure signals to target improvements instead of relying only on final task outcomes.

Read on huggingface.co →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.