10 papers
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1
Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reasoning performance to collapse o…
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Vincent-Daniel Yun, Woosang Lim, Haneul Yoo +3
Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared con…
KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving
Minsoo Cheong, Woosang Lim, Vincent-Daniel Yun +1
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this…
Output-Aware Rotation for INT2 KV-Cache Quantization
Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4
The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important…
Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3
Depth pruning improves the inference efficiency of large language models by removing Transformer blocks. Prior work typically treats layer redundancy as an inherent structural prop…
Locality-Aware Redundancy Pruning for LLM Depth Compression
Vincent-Daniel Yun, Youngrae Kim, Woosang Lim +3
Large language models are known to contain representational redundancy across network depth, making depth pruning an effective approach for improving inference efficiency. Existing…
EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
Minsoo Cheong, Donghyun Son, Woosang Lim +1
Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing…
Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
Euntae Choi, Sumin Song, Woosang Lim +1
Large Language Models (LLMs) face deployment challenges due to high computational costs, and while Post-Training Quantization (PTQ) offers a solution, existing rotation-based metho…
Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer
Euntae Choi, Sumin Song, Woosang Lim +1
We propose Rotate, Clip, and Partition (RCP), a quantization-aware training (QAT) approach that first realizes extreme compression of LLMs with W2A4KV4(2-bit weight, 4-bit activati…
Efficient and Scalable Estimation of Tool Representations in Vector Space
Suhong Moon, Siddharth Jha, Lutfi Eren Erdogan +4
Recent advancements in function calling and tool use have significantly enhanced the capabilities of large language models (LLMs) by enabling them to interact with external informa…