12 papers
Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training
Zezhou Wang, Hanzhang Zhou, Panrong Tong +5
Scaling GUI reinforcement learning requires environments that preserve real-app behavior. Yet coupling behavior inference with coding can turn partial traces into runnable mocks wi…
OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use
Hanzhang Zhou, Long Li, Quyu Kong +12
Computer use has reached a turning point: the focus is expanding from GUI-only interaction to systems that complete tasks on users' computers through visual and programmatic interf…
MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
Panrong Tong, Quyu Kong, Pengxiang Li +12
Online reinforcement learning (RL) with verifiable rewards has driven rapid progress in mathematics, coding, and tool use, but scaling this success to GUI agents remains challengin…
From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks
Yuchen Sun, Chenglin Cai, Gongjie Zhang +8
Mobile GUI agents complete tasks using GUI actions like taps and swipes. These actions are broadly applicable across applications, but reaching a navigation interface. A single dee…
Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
Hanzhang Zhou, Panrong Tong, Xu Zhang +13
GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on…
What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States
Chen Liu, Ling Chen, Hanzhang Zhou +7
Mobile GUI agents increasingly face long-horizon tasks that require reading, updating, and reusing task-relevant data across pages and applications. Existing methods treat memory l…
Bridging Domain Expertise and Generalization for Performance Estimation
Shuxuan Li, Zhilin Zhao, Quyu Kong +1
Performance estimation under distribution shift aims to predict how a model behaves on an unlabeled test set whose distribution differs from the training data, a scenario that requ…
De-Linearizing Agent Traces: Bayesian Inference of Latent Partial Orders for Efficient Execution
Dongqing Li, Zheqiao Cheng, Geoff K. Nicholls +1
AI agents increasingly execute procedural workflows as sequential action traces, which obscures latent concurrency and induces repeated step-by-step reasoning. We introduce BPOP, a…
MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments
Quyu Kong, Xu Zhang, Zhenyu Yang +10
Among existing online mobile-use benchmarks, AndroidWorld has emerged as the dominant benchmark due to its reproducible environment and deterministic evaluation; however, recent ag…
MAI-UI Technical Report: Real-World Centric Foundation GUI Agents
Hanzhang Zhou, Xu Zhang, Panrong Tong +8
The development of GUI agents could revolutionize the next generation of human-computer interaction. Motivated by this vision, we present MAI-UI, a family of foundation GUI agents…
UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning
Liangyu Chen, Hanzhang Zhou, Chenglin Cai +10
GUI grounding, which maps natural-language instructions to actionable UI elements, is a core capability of GUI agents. Prior works largely treats instructions as a static proxy for…
Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions
He Zhu, Quyu Kong, Kechun Xu +5
Grounding 3D object affordance is a task that locates objects in 3D space where they can be manipulated, which links perception and action for embodied intelligence. For example, f…