8 citations · 11 across the 4 of their papers we have counts for
6 papers
On Subquadratic Architectures: From Applications to Principles
Anamaria-Roberta Hartl, Levente Zólyomi, David Stap +6
Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost. Subquadratic architectures offer a scalable alternative. Howeve…
Effective Distillation to Hybrid xLSTM Architectures
Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7
There have been numerous attempts to distill quadratic attention-based large language models (LLMs) into sub-quadratic linearized architectures. However, despite extensive research…
Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences
Niklas Schmidinger, Lisa Schneckenreiter, Philipp Seidl +7
Language models for biological and chemical sequences enable crucial applications such as drug discovery, protein engineering, and precision medicine. Currently, these language mod…
Principled Weight Initialisation for Input-Convex Neural Networks
Pieter-Jan Hoedt, Günter Klambauer
Input-Convex Neural Networks (ICNNs) are networks that guarantee convexity in their input-output mapping. These networks have been successfully applied for energy-based modelling,…
MC-LSTM: Mass-Conserving LSTM
Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz +5
The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related…
Using LSTMs for climate change assessment studies on droughts and floods
Frederik Kratzert, Daniel Klotz, Johannes Brandstetter +3
Climate change affects occurrences of floods and droughts worldwide. However, predicting climate impacts over individual watersheds is difficult, primarily because accurate hydrolo…