6 papers
MobileViews: A Million-scale and Diverse Mobile GUI Dataset
Longxi Gao, Li Zhang, Shihe Wang +6
Visual language models (VLMs) empower mobile GUI agents to interpret complex mobile screens and respond to user requests. Training such capable agents requires large-scale, high-qu…
GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning
Longxi Gao, Li Zhang, Pengzhi Gao +3
Training effective Vision-Language Models (VLMs) for GUI agents typically depends on large-scale annotated datasets, whose collection is both labor-intensive and error-prone. We in…
GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration
Yuchen Sun, Shanhui Zhao, Tao Yu +5
GUI agents hold significant potential to enhance the experience and efficiency of human-device interaction. However, current methods face challenges in generalizing across applicat…
ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents
Haiyang Shen, Yue Li, Desong Meng +5
Recent advancements in integrating large language models (LLMs) with application programming interfaces (APIs) have gained significant interest in both academia and industry. Recen…
DroidCall: A Dataset for LLM-powered Android Intent Invocation
Weikai Xie, Li Zhang, Shihe Wang +2
The growing capabilities of large language models in natural language understanding significantly strengthen existing agentic systems. To power performant on-device mobile agents f…
A First Look at GPT Apps: Landscape and Vulnerability
Zejun Zhang, Li Zhang, Xin Yuan +3
Following OpenAI's introduction of GPTs, a surge in GPT apps has led to the launch of dedicated LLM app stores. Nevertheless, given its debut, there is a lack of sufficient underst…