1 citations · 1 across the 4 of their papers we have counts for
5 papers
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
Fei Wang, Chao Xue, Taoran Liu +3
Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon tha…
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
Fei Wang, Li Shen, Liang Ding +3
Zeroth-Order optimization presents a promising memory-efficient paradigm for fine-tuning Large Language Models by relying solely on forward passes. However, its practical adoption…
Layer as Puzzle Pieces: Compressing Large Language Models through Layer Concatenation
Fei Wang, Li Shen, Liang Ding +3
Large Language Models excel at natural language processing tasks, but their massive size leads to high computational and storage demands. Recent works have sought to reduce their m…
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
Fei Wang, Li Shen, Liang Ding +3
Fine-tuning is powerful for adapting large language models to downstream tasks, but it often results in huge memory usages. A promising approach to mitigate this is using Zeroth-Or…
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?
Fei Wang, Liang Ding, Jun Rao +3
The multimedia community has shown a significant interest in perceiving and representing the physical world with multimodal pretrained neural network models, and among them, the vi…