8 papers · 1 filter
Closed-Form Spectral Regularization for Multi-Task Model Merging
Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4
Model merging combines several independently fine-tuned experts into a single multi-task model without any training data, reducing the storage, serving, and decentralized-developme…
Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
Zixuan Hu, Li Shen, Zhenyi Wang +2
Harmful fine-tuning poses critical safety risks to fine-tuning-as-a-service for large language models. Existing defense strategies preemptively build robustness via attack simulati…
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors
Runxi Cheng, Feng Xiong, Yongxian Wei +2
Model merging seeks to integrate task-specific expert models into a unified architecture while preserving multi-task generalization capabilities, yet parameter interference between…
Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent
Yongxian Wei, Anke Tang, Li Shen +3
Merging multiple expert models offers a promising approach for performing multi-task learning without accessing their original data. Existing methods attempt to alleviate task conf…
Learn To Learn More Precisely
Runxi Cheng, Yongxian Wei, Xianglong He +5
Meta-learning has been extensively applied in the domains of few-shot learning and fast adaptation, achieving remarkable performance. While Meta-learning methods like Model-Agnosti…
Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models
Yongxian Wei, Zixuan Hu, Li Shen +4
Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data, enabling the rapid adaptation to new unseen t…
FREE: Faster and Better Data-Free Meta-Learning
Yongxian Wei, Zixuan Hu, Zhenyi Wang +3
Data-Free Meta-Learning (DFML) aims to extract knowledge from a collection of pre-trained models without requiring the original data, presenting practical benefits in contexts cons…
Task-Distributionally Robust Data-Free Meta-Learning
Zixuan Hu, Yongxian Wei, Li Shen +4
Data-Free Meta-Learning (DFML) aims to enable efficient learning of unseen few-shot tasks, by meta-learning from multiple pre-trained models without accessing their original traini…