15 papers
GUI-AC: Enhancing Continual Learning in GUI Agents
Can Lin, Wenjun Yang, Tao Feng +6
Graphical User Interfaces (GUIs) serve as the dominant medium for human-computer interaction, yet building GUI agents that generalize across the vast diversity of real-world interf…
Generative-Model Predictive Planning for Navigation in Partially Observable Environments
Thomas Quilter, Yifan Zhu, Guorui Quan +2
Navigation in partially observable environments presents a significant challenge for autonomous agents, requiring effective decision-making with limited sensory information in unkn…
Directory-Aware Query and Maintenance in Vector Databases
Mengzhao Wang, Zheng Gong, Jingpei Hu +4
Vector databases typically manage metadata as flat scalar attributes, which limits their ability to express hierarchical directory semantics commonly used to organize code reposito…
VikingMem: A Memory Base Management System for Stateful LLM-based Applications
Jiajie Fu, Junwen Chen, Mengzhao Wang +5
Large Language Models have revolutionized interactive applications; however, their finite context windows pose a critical data management challenge for maintaining stateful, long-t…
5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning
Yifan Zhu, Can Lin, Hangjie Yuan +4
Parameter-Efficient Fine-Tuning (PEFT) methods provide a streamlined and efficient tool for adapting large models to domain-specific multimodal downstream tasks. Although these met…
An Information-Theoretic Definition for Open-Ended Learning
Wanqiao Xu, Yifan Zhu, Benjamin Van Roy
A growing body of work points to the great promise of AI systems that can continually expand their capabilities as they operate in an open-ended environment. But yet there is no co…
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
Zizhuo Lin, Quanling Liu, Jinsheng Quan +6
Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a cl…
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
Mengyang Sun, Maochuan Dou, Tao Feng +5
While Large Language Models (LLMs) are commonly fine-tuned to handle domain-specific tasks before being applied to vertical applications, adapting them to complex scenarios with di…
HEDP: A Hybrid Energy-Distance Prompt-based Framework for Domain Incremental Learning
Yu Feng, Zhen Tian, Haoran Luo +8
Domain Incremental Learning is a critical scenario that requires models to continuously adapt to new data domains without retraining. However, domain shifts often cause severe perf…
RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
Hao Gao, Shaoyu Chen, Yifan Zhu +4
High-level autonomous driving requires motion planners capable of modeling multimodal future uncertainties while remaining robust in closed-loop interactions. Although diffusion-ba…
Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning
Yuehao Song, Shaoyu Chen, Hao Gao +8
Vision-language models (VLMs) enhance the planning capability of end-to-end (E2E) driving policy by leveraging high-level semantic reasoning. However, existing approaches often ove…
Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information
Yifan Zhu, Yibo Miao, Yinpeng Dong +1
The volume of freely scraped data on the Internet has driven the tremendous success of deep learning. Along with this comes the growing concern about data privacy and security. Num…
Continual GUI Agents
Ziwei Liu, Borui Kang, Hangjie Yuan +4
As digital environments (data distribution) are in flux, with new GUI data arriving over time-introducing new domains or resolutions-agents trained on static environments deteriora…
C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning
Wei Li, Hangjie Yuan, Zixiang Zhao +4
Balancing sensitivity to new tasks and stability for retaining past knowledge is crucial in continual learning (CL). Recently, sharpness-aware minimization has proven effective in…
Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Models
Ziwei Liu, Borui Kang, Wei Li +6
Vision-Language Continual Learning (VLCL) has attracted significant research attention for its robust capabilities, and the adoption of Parameter-Efficient Fine-Tuning (PEFT) strat…