14 papers
Tailoring the Quantization Space for 1-Bit KV Cache Compression
Minsoo Cheong, Donghyun Son, Sungjoo Yoo
The key-value (KV) cache becomes a major memory bottleneck in long-context LLM inference, placing substantial pressure on memory capacity and bandwidth. To mitigate this bottleneck…
LiteEvo: Automated, Cost-Efficient Harness Evolution for Generalization to Unseen Tasks
Euntae Choi, Sumin Song, Sungjoo Yoo
An LLM agent is defined by two things: the weights inside its model and the harness of components assembled around it. Harnesses are still handcrafted, and HarnessX, which evolves…
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Vincent-Daniel Yun, Woosang Lim, Haneul Yoo +3
Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared con…
KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving
Minsoo Cheong, Woosang Lim, Vincent-Daniel Yun +1
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this…
Output-Aware Rotation for INT2 KV-Cache Quantization
Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4
The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important…
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
Euntae Choi, Sumin Song, Sungjoo Yoo
Large reasoning models (LRMs) reach competition-level math and coding accuracy via long autoregressive decoding, making per-token decoding cost a primary deployment concern. Weight…
EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
Minsoo Cheong, Donghyun Son, Woosang Lim +1
Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing…
Gaussian Weight Sampling for Scalable, Efficient and Stable Pseudo-Quantization Training
Myeonghwan Ahn, Sungjoo Yoo
Ever-growing scale of large language models (LLMs) is pushing for improved efficiency, favoring fully quantized training (FQT) over BF16. While FQT accelerates training, it faces c…
NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
Donghyun Son, Euntae Choi, Sungjoo Yoo
Large Language Model (LLM) inference is typically memory-intensive, especially when processing large batch sizes and long sequences, due to the large size of key-value (KV) cache.…
Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
Euntae Choi, Sumin Song, Woosang Lim +1
Large Language Models (LLMs) face deployment challenges due to high computational costs, and while Post-Training Quantization (PTQ) offers a solution, existing rotation-based metho…
Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer
Euntae Choi, Sumin Song, Woosang Lim +1
We propose Rotate, Clip, and Partition (RCP), a quantization-aware training (QAT) approach that first realizes extreme compression of LLMs with W2A4KV4(2-bit weight, 4-bit activati…
Phys3DGS: Physically-based 3D Gaussian Splatting for Inverse Rendering
Euntae Choi, Sungjoo Yoo
We propose two novel ideas (adoption of deferred rendering and mesh-based representation) to improve the quality of 3D Gaussian splatting (3DGS) based inverse rendering. We first r…
Baking Relightable NeRF for Real-time Direct/Indirect Illumination Rendering
Euntae Choi, Vincent Carpentier, Seunghun Shin +1
Relighting, which synthesizes a novel view under a given lighting condition (unseen in training time), is a must feature for immersive photo-realistic experience. However, real-tim…
Breaking MLPerf Training: A Case Study on Optimizing BERT
Yongdeok Kim, Jaehyung Ahn, Myeongwoo Kim +11
Speeding up the large-scale distributed training is challenging in that it requires improving various components of training including load balancing, communication, optimizers, et…