5 papers
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5
Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on…
Effective Distillation to Hybrid xLSTM Architectures
Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7
There have been numerous attempts to distill quadratic attention-based large language models (LLMs) into sub-quadratic linearized architectures. However, despite extensive research…
On Subquadratic Architectures: From Applications to Principles
Anamaria-Roberta Hartl, Levente Zólyomi, David Stap +6
Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost. Subquadratic architectures offer a scalable alternative. Howeve…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not used
Srijan Shakya, Anamaria-Roberta Hartl, Sepp Hochreiter +1
Large Language Models (LLMs) often falter in complex reasoning tasks due to their static, parametric knowledge, leading to hallucinations and poor performance in specialized domain…