6 papers
Energy-Structured Low-Rank Adaptation for Continual Learning
Longhua Li, Lei Qi, Qi Tian +1
While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compac…
Model Merging in the Essential Subspace
Longhua Li, Lei Qi, Qi Tian +1
Model merging aims to integrate multiple task-specific fine-tuned models derived from a shared pre-trained checkpoint into a single multi-task model without additional training. De…
Enhancing Post-Training Quantization via Future Activation Awareness
Zheqi Lv, Zhenxuan Fan, Qi Tian +2
Post-training quantization (PTQ) is a widely used method to compress large language models (LLMs) without fine-tuning. It typically sets quantization hyperparameters (e.g., scaling…
Tackling Device Data Distribution Real-time Shift via Prototype-based Parameter Editing
Zheqi Lv, Wenqiao Zhang, Kairui Fu +6
The on-device real-time data distribution shift on devices challenges the generalization of lightweight on-device models. This critical issue is often overlooked in current researc…
Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion
Zheqi Lv, Junhao Chen, Qi Tian +3
Diffusion models have become the mainstream architecture for text-to-image generation, achieving remarkable progress in visual quality and prompt controllability. However, current…
Optimize Incompatible Parameters through Compatibility-aware Knowledge Integration
Zheqi Lv, Keming Ye, Zishu Wei +7
Deep neural networks have become foundational to advancements in multiple domains, including recommendation systems, natural language processing, and so on. Despite their successes…