21 papers
HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learning
Zhen-Hao Xie, Yan Wang, Lan Li +3
Class-Incremental Learning (CIL) aims to endow models with the ability to continuously adapt to evolving data streams. Recent advances in pre-trained vision-language models (e.g.,…
AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
Zhen-Hao Xie, Yu-Cheng Shi, Da-Wei Zhou
Class-Incremental Learning (CIL) is important in building real-world learning systems. In CLIP-based CIL, the model performs classification by comparing similarity between visual a…
SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
Zhen-Hao Xie, Jun-Tao Tang, Yu-Cheng Shi +3
Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to continually expand their capabilities, ma…
Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning
Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie +1
Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruction tuning. However, real-wor…
Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
Jiayu Li, Enpei Zhang, Dawei Zhou +2
We study workflow learning in a setting where specialized agents hand off control through a shared artifact, each agent observes only a local function of that artifact and its own…
Stable Routing for Mixture-of-Experts in Class-Incremental Learning
Zirui Guo, Quan Cheng, Da-Wei Zhou +1
Class-incremental learning (CIL) requires models to learn new classes sequentially while preserving prior knowledge. Recently, approaches that combine pre-trained models with mixtu…