10 papers
JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
Zixiang Chen, Yuheng Lu, Zihao Cheng +8
Real-world GUI usage frequently involves workflows that span multiple devices and platforms, requiring the transfer of intermediate results, maintenance of shared state, and coordi…
Towards Generalizable Visually Grounded Exploration of Household Devices
Linhao Zheng, Zeming Liu, Wangke Chen +4
Recent advancements in Vision-Language Models (VLMs) have demonstrated impressive capabilities in static visual recognition and high-level semantic reasoning. However, current embo…
Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory
Zihao Cheng, Yingyu Shan, Hongru Wang +6
Travel planning agents assist users in generating personalized travel plans by modeling their individual preferences. Existing agents either rely on explicit user instructions or e…
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
Yingyu Shan, Yuhang Guo, Zihao Cheng +7
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform c…
PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes
Yingyu Shan, Zeming Liu, Silin Li +4
Recent advancements in Large Language Models (LLMs) have empowered home assistants with natural language interaction capabilities. However, current assistants overlook the progress…
Benchmarking Living-Screen-Native GUI Agents on Short-Video Platforms
Jiashu Yao, Heyan Huang, Daiqing Wu +5
GUI agents today assume a static screen, where the world is frozen between two actions. However, real interfaces such as short-video applications violate this assumption, as their…
Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
Zihao Cheng, Hongru Wang, Zeming Liu +6
Terminal agents extend Large Language Models with the ability to execute tasks directly in command-line environments, but their progress is bottlenecked by the scarcity of high-qua…
DocOS: Towards Proactive Document-Guided Actions in GUI Agents
Jingjing Liu, Ziye Huang, Zihao Cheng +6
While Graphical User Interface (GUI) agents have shown promising performance in automated device interaction, they primarily depend on static parametric knowledge from pre-training…
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst
Hongru Wang, Deng Cai, Wanjun Zhong +4
Inference-time scaling has attracted much attention which significantly enhance the performance of Large Language Models (LLMs) in complex reasoning tasks by increasing the length…
Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges
Hongru Wang, Wenyu Huang, Yufei Wang +7
Existing benchmarks that assess Language Models (LMs) as Language Agents (LAs) for tool use primarily focus on stateless, single-turn interactions or partial evaluations, such as t…