11 papers
Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models
Yang Chen, Zhan Zhuang, Yanbin Wei +3
While adversarial prompt tuning can enhance robustness of vision-language models efficiently, we find that existing methods aggravate robust generalization overfitting on seen clas…
Curvature-Guided Mixing for MLLM Adaptation
Jinglong Yang, Jiaxuan He, Wenjian Huang +2
Fine-tuning Multimodal Large Language Models (MLLMs) on specialized tasks often leads to catastrophic forgetting of their general capabilities. Existing model merging methods to co…
Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspective
Zhichao Chen, Zhan Zhuang, Yunfei Teng +6
Gradual domain adaptation (GDA) aims to mitigate domain shift by progressively adapting models from the source domain to the target domain via intermediate domains. However, real i…
NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs
Shuaidi Wang, Zhan Zhuang, Ruping Huang +1
Diffusion Large Language Models (dLLMs) have emerged as a promising non-autoregressive generative paradigm. Given the prohibitive computational cost of full fine-tuning, Parameter-…
Beyond Uniform Credit Assignment: Selective Eligibility Traces for RLVR
Chaoli Mou, Zhan Zhuang, Xinning Chen +1
Reinforcement Learning with Verifiable Rewards (RLVR) has become a key approach for improving the reasoning abilities of large language models. However, widely used critic-free alg…
HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learning
Xuerui Zhang, Xuehao Wang, Zhan Zhuang +5
Lifelong learning aims to preserve knowledge acquired from previous tasks while incorporating knowledge from a sequence of new tasks. However, most prior work explores only streams…