5 papers
StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents
Yan Yang, Xiangru Jian, Ziyang Luo +7
Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering…
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Tianyu Xie, Jinfa Huang, Yuexiao Ma +11
Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text. However, existing OLM benchmarks remain anchored to stat…
Benchmarking and Improving GUI Agents in High-Dynamic Environments
Enqi Liu, Liyuan Pan, Zhi Gao +5
Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and reinforcement learning (RL). How…
Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
Chunqiu Steven Xia, Zhe Wang, Yan Yang +2
Large Language Models (LLMs) are reshaping almost all industries, including software engineering. In recent years, a number of LLM agents have been proposed to solve real-world sof…
GTA1: GUI Test-time Scaling Agent
Yan Yang, Dongxu Li, Yutong Dai +12
Graphical user interface (GUI) agents autonomously complete tasks across platforms (\eg, Linux) by sequentially decomposing user instructions into action proposals that iteratively…