6 papers
Task-Adaptive Rubrics for GUI Reward Modeling
Tao Xiong, Xavier Hu, Wenkai Wang +6
Recent studies on GUI agents have increasingly focused on outcome reward modeling, which assigns outcome rewards by judging whether an executed trajectory satisfies the success cri…
Xiaomi-GUI-0 Technical Report
Wanxia Cao, Chengzhen Duan, Pei Fu +29
Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, tex…
DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration
Wenkai Wang, Tao Xiong, Jingchen Ni +6
Real-world professional desktop workflows in specialized creative and engineering software unfold over long horizons and often require human-in-the-loop coordination, where agents…
GUI-PRA: Process Reward Agent for GUI Tasks
Tao Xiong, Xavier Hu, Yurun Chen +6
Graphical User Interface (GUI) Agents powered by Multimodal Large Language Models (MLLMs) show significant potential for automating tasks. However, they often struggle with long-ho…
OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use
Xueyu Hu, Tao Xiong, Biao Yi +26
The dream to create AI assistants as capable and versatile as the fictional J.A.R.V.I.S from Iron Man has long captivated imaginations. With the evolution of (multi-modal) large la…
Mixture of Reasonings: Teach Large Language Models to Reason with Adaptive Strategies
Tao Xiong, Xavier Hu, Wenyan Fan +1
Large language models (LLMs) excel in complex tasks through advanced prompting techniques like Chain-of-Thought (CoT) and Tree-of-Thought (ToT), but their reliance on manually craf…