1 citations · 1 across the 6 of their papers we have counts for
9 papers
ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
Heejun Kim, Junyoung Lee, SangLyul Cho +3
Looped Transformers improve parameter efficiency by repeatedly applying shared Transformer blocks over multiple recurrent loops, increasing computational depth without increasing t…
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
SangLyul Cho, Langqing Cui, Sehoon Kim +2
Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurrent depths. However, they incur…
HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion
Dongyeun Lee, Amir Zandieh, Vahab Mirrokni +2
Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation. However, their ability to produce long-duration videos is fundame…
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem
Seongjin Cha, Gyuwan Kim, Dongsu Han +2
Self-speculative decoding (SSD) accelerates LLM inference by skipping layers to create an efficient draft model, yet existing methods often rely on static heuristics that ignore th…
DNACHUNKER: Learnable Tokenization for DNA Language Models
Taewon Kim, Jihwan Shin, Hyomin Kim +5
DNA language models are increasingly used to represent genomic sequence, yet their effectiveness depends critically on how raw nucleotides are converted into model inputs. Unlike n…
CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
Insu Han, Zeliang Zhang, Zhiyuan Wang +8
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance across diverse applications. However, their computational overhead during deployment remains a cri…
PolarQuant: Quantizing KV Caches with Polar Transformation
Insu Han, Praneeth Kacham, Amin Karbasi +2
Large language models (LLMs) require significant memory to store Key-Value (KV) embeddings in their KV cache, especially when handling long-range contexts. Quantization of these KV…
Streaming Attention Approximation via Discrepancy Theory
Ekaterina Kochetkova, Kshiteej Sheth, Insu Han +2
Large language models (LLMs) have achieved impressive success, but their high memory requirements present challenges for long-context token generation. In this paper we study the s…
SubGen: Token Generation in Sublinear Time and Memory
Amir Zandieh, Insu Han, Vahab Mirrokni +1
Despite the significant success of large language models (LLMs), their extensive memory requirements pose challenges for deploying them in long-context token generation. The substa…