2 citations · 9 across the 20 of their papers we have counts for
21 papers
K^2-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control
Zhe Wu, Donglin Mo, Hongjin Lu +7
Existing mobile device control agents often perform poorly when solving complex tasks requiring long-horizon planning and precise operations, typically due to a lack of relevant ta…
ActionCodec: What Makes for Good Action Tokenizers
Zibin Dong, Yicheng Liu, Shiduo Zhang +8
Vision-Language-Action (VLA) models leveraging the native autoregressive paradigm of Vision-Language Models (VLMs) have demonstrated superior instruction-following and training eff…
Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning
Yuxuan Zhou, Yubin Wang, Bin Wang +4
Large language models (LLMs) have shown great promise in the medical domain, achieving strong performance on several benchmarks. However, they continue to underperform in real-worl…
Hi-Agent: Hierarchical Vision-Language Agents for Mobile Device Control
Zhe Wu, Hongjin Lu, Junliang Xing +10
Building agents that autonomously operate mobile devices has attracted increasing attention. While Vision-Language Models (VLMs) show promise, most existing approaches rely on dire…
Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control
Georgios Papoudakis, Thomas Coste, Jianye Hao +2
Reinforcement learning (RL) using foundation models for policy approximations in multi-turn tasks remains challenging. We identify two main limitations related to sparse reward set…
Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding
Jian Hu, Zixu Cheng, Shaogang Gong +4
Video Temporal Grounding (TG) aims to temporally locate video segments matching a natural language description (a query) in a long video. While Vision-Language Models (VLMs) are ef…
Collab-Solver: Collaborative Solving Policy Learning for Mixed-Integer Linear Programming
Siyuan Li, Yifan Yu, Zhihao Zhang +5
Mixed-integer linear programming (MILP) has been a fundamental problem in combinatorial optimization. Conventional MILP solving mainly relies on carefully designed heuristics embed…
Deep Research Agents: A Systematic Examination And Roadmap
Yuxuan Huang, Yihang Chen, Haozheng Zhang +10
The rapid progress of Large Language Models (LLMs) has given rise to a new category of autonomous AI systems, referred to as Deep Research (DR) agents. These agents are designed to…
Beyond Syntax: Action Semantics Learning for App Agents
Bohan Tang, Dezhao Luo, Jianheng Liu +5
The recent development of Large Language Models (LLMs) enables the rise of App agents that interpret user intent and operate smartphone Apps through actions such as clicking and sc…
AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search
Yu Li, Lehui Li, Zhihao Wu +5
Large language model (LLM) agents have demonstrated strong capabilities across diverse domains, yet automated agent design remains a significant challenge. Current automated agent…
ViMo: A Generative Visual GUI World Model for App Agents
Dezhao Luo, Bohan Tang, Kang Li +6
App agents, which autonomously operate mobile Apps through Graphical User Interfaces (GUIs), have gained significant interest in real-world applications. Yet, they often struggle w…
AppVLM: A Lightweight Vision Language Model for Online App Control
Georgios Papoudakis, Thomas Coste, Zhihao Wu +3
The utilisation of foundation models as smartphone assistants, termed app agents, is a critical research challenge. These agents aim to execute human instructions on smartphones by…
Generative Models in Decision Making: A Survey
Xinyu Shao, Jianping Zhang, Haozhi Wang +9
Generative models have fundamentally reshaped the landscape of decision-making, reframing the problem from pure scalar reward maximization to high-fidelity trajectory generation an…
Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learning
Qingyuan Wu, Jianheng Liu, Jianye Hao +2
State-of-the-art (SOTA) reinforcement learning (RL) methods have enabled vision-language model (VLM) agents to learn from interaction with online environments without human supervi…
Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization
Yongcheng Zeng, Xinyu Cui, Xuanfa Jin +11
Self-Refinement refers to a model's ability to revise its own responses to produce improved outputs. This capability can also serve as a fundamental mechanism for Self-Improvement,…
Trajectory World Models for Heterogeneous Environments
Shaofeng Yin, Jialong Wu, Siqiao Huang +4
Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor info…
GUI Agents with Foundation Models: A Comprehensive Survey
Shuai Wang, Weiwen Liu, Jingxuan Chen +12
Recent advances in foundation models, particularly Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs), have facilitated the development of intelligent agents…
Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance
Antoine Grosnit, Alexandre Maraval, Refinath S N +16
Human expertise emerges through iterative cycles of interaction, reflection, and internal model updating, which are central to cognitive theories such as Kolb's experiential learni…
Lightweight Neural App Control
Filippos Christianos, Georgios Papoudakis, Thomas Coste +3
This paper introduces a novel mobile phone control architecture, Lightweight Multi-modal App Control (LiMAC), for efficient interactions and control across various Android apps. Li…
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
Jingxuan Chen, Derek Yuen, Bin Xie +14
Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contender…
DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents
Taiyi Wang, Zhihao Wu, Jianheng Liu +3
On-device control agents, especially on mobile devices, are responsible for operating mobile devices to fulfill users' requests, enabling seamless and intuitive interactions. Integ…