5 papers
Automatic Pruning Discovery for Large Language Models
Haidong Kang, Lihong Lin, Enneng Yang +2
Large language models (LLMs) have achieved remarkable performance on a wide range of tasks, hindering real-world deployment due to their massive size. Existing pruning methods (e.g…
Unlocking the Potential of Continual Model Merging: An ODE Perspective
Lihong Lin, Haidong Kang
Continual Model Merging (CMM) enables rapid customization of foundation models by sequentially incorporating task-adapted models without repeated retraining. However, existing merg…
Revolutionizing Mixed Precision Quantization: Towards Training-free Automatic Proxy Discovery via Large Language Models
Haidong Kang, Jun Du, Lihong Lin
Mixed-Precision Quantization (MPQ) liberates Deep Neural Networks (DNNs) from the Out-Of-Memory (OOM) bottleneck and has garnered increasing research attention. However, convention…
Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusion
Haidong Kang, Ketong Qian, Yi Lu
Efforts to overcome catastrophic forgetting in Few-Shot Class-Incremental Learning (FSCIL) have primarily focused on developing more effective gradient-based optimization strategie…
QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs
Taolin Zhang, Haidong Kang, Dongyang Li +3
Recently, large language models (LLMs) have demonstrated impressive results but still suffer from hallucinations. Model editing has been proposed to correct factual inaccuracies in…