5 papers
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5
Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on…
On Subquadratic Architectures: From Applications to Principles
Anamaria-Roberta Hartl, Levente Zólyomi, David Stap +6
Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost. Subquadratic architectures offer a scalable alternative. Howeve…
Effective Distillation to Hybrid xLSTM Architectures
Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7
There have been numerous attempts to distill quadratic attention-based large language models (LLMs) into sub-quadratic linearized architectures. However, despite extensive research…
Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not used
Srijan Shakya, Anamaria-Roberta Hartl, Sepp Hochreiter +1
Large Language Models (LLMs) often falter in complex reasoning tasks due to their static, parametric knowledge, leading to hallucinations and poor performance in specialized domain…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…