5 papers
Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models
Rayyan Abdalla, Amir Hussein, Min Wu +1
Post-training quantization (PTQ) is critical for the efficient deployment of large language models (LLMs). Recent ultra-low-bit PTQ methods rely on rigid weight-saliency assumption…
CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
Jingze Ge, Xue Geng, Yun Liu +6
To mitigate the memory constraints associated with fine-tuning large pre-trained models, existing parameter-efficient fine-tuning (PEFT) methods, such as LoRA, rely on low-rank upd…
Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces
Jingze Ge, Yun Liu, Xue Geng +4
Adapting large pretrained models to diverse tasks is now routine, yet the two dominant strategies of parameter-efficient fine-tuning (PEFT) and low-rank compression are typically c…
DM3D: Distortion-Minimized Weight Pruning for Lossless 3D Object Detection
Kaixin Xu, Qingtian Feng, Hao Chen +6
Applying deep neural networks to 3D point cloud processing has attracted increasing attention due to its advanced performance in many areas, such as AR/VR, autonomous driving, and…
LPViT: Low-Power Semi-structured Pruning for Vision Transformers
Kaixin Xu, Zhe Wang, Chunyun Chen +7
Vision transformers have emerged as a promising alternative to convolutional neural networks for various image analysis tasks, offering comparable or superior performance. However,…