5 papers
Towards Scalable Lightweight GUI Agents via Multi-role Orchestration
Ziwei Wang, Junjie Zheng, Leyang Yang +7
Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) enable digital automation on end-user devices. While scaling both parameters an…
ProBench: Benchmarking GUI Agents with Accurate Process Information
Leyang Yang, Ziwei Wang, Xiaoxuan Tang +4
With the deep integration of artificial intelligence and interactive technology, Graphical User Interface (GUI) Agent, as the carrier connecting goal-oriented natural language and…
History-Aware Reasoning for GUI Agents
Ziwei Wang, Leyang Yang, Xiaoxuan Tang +4
Advances in Multimodal Large Language Models have significantly enhanced Graphical User Interface (GUI) automation. Equipping GUI agents with reliable episodic reasoning capabiliti…
PG-Agent: An Agent Powered by Page Graph
Weizhi Chen, Ziwei Wang, Leyang Yang +5
Graphical User Interface (GUI) agents possess significant commercial and social value, and GUI agents powered by advanced multimodal large language models (MLLMs) have demonstrated…
MP-GUI: Modality Perception with MLLMs for GUI Understanding
Ziwei Wang, Weizhi Chen, Leyang Yang +7
Graphical user interface (GUI) has become integral to modern society, making it crucial to be understood for human-centric systems. However, unlike natural images or documents, GUI…