5 papers
Tailoring the Quantization Space for 1-Bit KV Cache Compression
Minsoo Cheong, Donghyun Son, Sungjoo Yoo
The key-value (KV) cache becomes a major memory bottleneck in long-context LLM inference, placing substantial pressure on memory capacity and bandwidth. To mitigate this bottleneck…
KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving
Minsoo Cheong, Woosang Lim, Vincent-Daniel Yun +1
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this…
Output-Aware Rotation for INT2 KV-Cache Quantization
Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4
The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important…
EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
Minsoo Cheong, Donghyun Son, Woosang Lim +1
Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing…
InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation
Junhyeong Hwangbo, Soohyun Lee, Hyeon Jeon +4
Even LLMs that appear safe during evaluation can still produce harmful responses in deployment. Because stochastic sampling yields different responses to the same prompt, low-proba…