6 papers
Output-Aware Rotation for INT2 KV-Cache Quantization
Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4
The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important…
Locality-Aware Redundancy Pruning for LLM Depth Compression
Vincent-Daniel Yun, Youngrae Kim, Woosang Lim +3
Large language models are known to contain representational redundancy across network depth, making depth pruning an effective approach for improving inference efficiency. Existing…
Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy +1
Layer pruning removes entire Transformer decoder blocks from large language models, but introduces a mismatch between the hidden state received by the next surviving layer and the…
Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3
Depth pruning improves the inference efficiency of large language models by removing Transformer blocks. Prior work typically treats layer redundancy as an inherent structural prop…
ZOO-Prune: Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models
Youngeun Kim, Youjia Zhang, Huiling Liu +3
Large Vision-Language Models (VLMs) enable strong multimodal reasoning but incur heavy inference costs from redundant visual tokens. Token pruning alleviates this issue, yet existi…
Dynamic Rank Adjustment for Accurate and Efficient Neural Network Training
Hyuntak Shin, Aecheon Jung, Sungeun Hong +1
Low-rank training methods reduce the number of trainable parameters by re-parameterizing the weights with matrix decompositions (e.g., singular value decomposition). However, enfor…