3 papers
cs.AI2026
ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions
Rui Xie, Lu Chen
Powerful code agents can execute scripts, call tools, and manage files, yet many important applications remain accessible primarily through graphical user interfaces. We argue that…
cs.AI2026
GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation
Rui Xie, Zhi Gao, Chenrui Shi +3
Large vision-language models have endowed GUI agents with strong general capabilities for interface understanding and interaction. However, due to insufficient exposure to domain-s…
cs.MA2024
MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task Automation
Zichen Zhu, Hao Tang, Yansi Li +13
Existing Multimodal Large Language Model (MLLM)-based agents face significant challenges in handling complex GUI (Graphical User Interface) interactions on devices. These challenge…