4.5k citations · 8.5k across the 90 of their papers we have counts for
11 papers · 1 filter
ParetoTransport: Generative Optimization by Mass Transport Toward The Pareto Front
Stephanie Holly, Sepp Hochreiter, Werner Zellinger
Offline multi-objective optimization requires not only moving the objective vectors of candidate designs toward the Pareto front, but also distributing them effectively along it. G…
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5
Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on…
Accelerating LLM Inference via Vector Index Based Output Embeddings
Martin Loretz, Sepp Hochreiter
Large output embedding matrices create a significant memory bandwidth bottleneck during autoregressive decoding, especially for compact LLMs with large multilingual vocabularies. W…
TiRex-2: Generalizing TiRex to Multivariate Data and Streaming
Patrick Podest, Marco Pichler, Elias Bürger +7
We introduce TiRex-2, a recurrent xLSTM-based time series foundation model that generalizes the univariate TiRex to multivariate forecasting with both past and future covariates. R…
On Subquadratic Architectures: From Applications to Principles
Anamaria-Roberta Hartl, Levente Zólyomi, David Stap +6
Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost. Subquadratic architectures offer a scalable alternative. Howeve…
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1
Recent advancements in reasoning language models have been driven by Reinforcement Learning (RL) fine-tuning. Most often, these rely on the Group Relative Policy Optimization (GRPO…
Unlocking the Working Memory of Large Language Models for Latent Reasoning
Lukas Aichberger, Sepp Hochreiter
To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this coupl…
Effective Distillation to Hybrid xLSTM Architectures
Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7
There have been numerous attempts to distill quadratic attention-based large language models (LLMs) into sub-quadratic linearized architectures. However, despite extensive research…
Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not used
Srijan Shakya, Anamaria-Roberta Hartl, Sepp Hochreiter +1
Large Language Models (LLMs) often falter in complex reasoning tasks due to their static, parametric knowledge, leading to hallucinations and poor performance in specialized domain…
AP-OOD: Attention Pooling for Out-of-Distribution Detection
Claus Hofmann, Christian Huber, Bernhard Lehner +3
Out-of-distribution (OOD) detection, which maps high-dimensional data into a scalar OOD score, is critical for the reliable deployment of machine learning models. A key challenge i…
The Offline-Frontier Shift: Diagnosing Distributional Limits in Generative Multi-Objective Optimization
Stephanie Holly, Alexandru-Ciprian Zăvoianu, Siegfried Silber +2
Offline multi-objective optimization (MOO) aims to recover Pareto-optimal designs given a finite, static dataset. Recent generative approaches, including diffusion models, show str…