1 citations · 1 across the 6 of their papers we have counts for
5 papers · 1 filter
ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
Heejun Kim, Junyoung Lee, SangLyul Cho +3
Looped Transformers improve parameter efficiency by repeatedly applying shared Transformer blocks over multiple recurrent loops, increasing computational depth without increasing t…
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
SangLyul Cho, Langqing Cui, Sehoon Kim +2
Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurrent depths. However, they incur…
HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion
Dongyeun Lee, Amir Zandieh, Vahab Mirrokni +2
Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation. However, their ability to produce long-duration videos is fundame…
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem
Seongjin Cha, Gyuwan Kim, Dongsu Han +2
Self-speculative decoding (SSD) accelerates LLM inference by skipping layers to create an efficient draft model, yet existing methods often rely on static heuristics that ignore th…
DNACHUNKER: Learnable Tokenization for DNA Language Models
Taewon Kim, Jihwan Shin, Hyomin Kim +5
DNA language models are increasingly used to represent genomic sequence, yet their effectiveness depends critically on how raw nucleotides are converted into model inputs. Unlike n…