7 papers
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
Bowen Yang, Kaiming Jin, Zhenyu Wu +12
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalizati…
InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition
Yijie Zheng, Weijie Wu, Qingyun Li +7
Language-Guided object recognition in remote sensing imagery is crucial for large-scale mapping and automated data annotation. However, existing open-vocabulary and visual groundin…
ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data
Zhaoyang Liu, Jingjing Xie, Zichen Ding +27
Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, o…
Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
Yabo Zhang, Yihan Zeng, Qingyun Li +3
Large language models (LLMs) have demonstrated strong capabilities in language understanding and reasoning, yet they remain limited when tackling real-world tasks that require up-t…
MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
Xuehui Wang, Zhenyu Wu, JingJing Xie +25
We introduce MMBench-GUI, a hierarchical benchmark for evaluating GUI automation agents across Windows, macOS, Linux, iOS, Android, and Web platforms. It comprises four levels: GUI…
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model
Wenke Huang, Jian Liang, Xianda Guo +14
Multi-modal Large Language Models (MLLMs) integrate visual and linguistic reasoning to address complex tasks such as image captioning and visual question answering. While MLLMs dem…