8 papers
MCP-Universe RL: A Framework for Training MCP Tool-Use Agents via Reinforcement Learning
Ziyang Luo, Yan Yang, Xiangru Jian +5
Reinforcement learning (RL) has become an effective way to improve the tool-use ability of large language models (LLMs), but most existing RL frameworks stop at the policy update.…
StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents
Yan Yang, Xiangru Jian, Ziyang Luo +7
Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering…
Benchmarking and Improving GUI Agents in High-Dynamic Environments
Enqi Liu, Liyuan Pan, Zhi Gao +5
Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and reinforcement learning (RL). How…
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Tianyu Xie, Jinfa Huang, Yuexiao Ma +11
Evaluating omni-modal large language models (OLMs) in multi-party dialogue requires more than answer correctness on pre-segmented inputs. We introduce SocialOmni, an offline diagno…
Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
Chunqiu Steven Xia, Zhe Wang, Yan Yang +2
Large Language Models (LLMs) are reshaping almost all industries, including software engineering. In recent years, a number of LLM agents have been proposed to solve real-world sof…
A Two-dimensional Zero-shot Dialogue State Tracking Evaluation Method using GPT-4
Ming Gu, Yan Yang
Dialogue state tracking (DST) is evaluated by exact matching methods, which rely on large amounts of labeled data and ignore semantic consistency, leading to over-evaluation. Curre…
Plan, Generate and Complicate: Improving Low-resource Dialogue State Tracking via Easy-to-Difficult Zero-shot Data Augmentation
Ming Gu, Yan Yang
Data augmentation methods have been a promising direction to improve the performance of small models for low-resource dialogue state tracking. However, traditional methods rely on…
State Value Generation with Prompt Learning and Self-Training for Low-Resource Dialogue State Tracking
Ming Gu, Yan Yang, Chengcai Chen +1
Recently, low-resource dialogue state tracking (DST) has received increasing attention. First obtaining state values then based on values to generate slot types has made great prog…