4 papers · 1 filter
Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold
Luca Benfenati, Matteo Risso, Andrea Vannozzi +5
Key-value (KV) caching enables fast autoregressive decoding but at long contexts becomes a dominant bottleneck in High Bandwidth Memory (HBM) capacity and bandwidth. A common mitig…
TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
Ahmet Caner Yüzügüler, Ahmet Çelik, Jiawei Zhuang +1
Multi-Head Latent Attention (MLA) is a recent attention mechanism adopted in state-of-the-art LLMs such as DeepSeek-v3 and Kimi K2. Thanks to its novel formulation, MLA allows two…
Flexible Channel Dimensions for Differentiable Architecture Search
Ahmet Caner Yüzügüler, Nikolaos Dimitriadis, Pascal Frossard
Finding optimal channel dimensions (i.e., the number of filters in DNN layers) is essential to design DNNs that perform well under computational resource constraints. Recent work i…
U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search
Ahmet Caner Yüzügüler, Nikolaos Dimitriadis, Pascal Frossard
Optimizing resource utilization in target platforms is key to achieving high performance during DNN inference. While optimizations have been proposed for inference latency, memory…