12 papers
LoopMTP: A looped transformer guided by latent multi-token prediction
Behzad Shomali, Markus Frey, David Berghaus +2
Looped transformers have emerged as a parameter-efficient alternative to scaling depth for strong reasoning. By reusing one stack of layers across iterations, they attain the e…
Foundation Inference Models for Ordinary Differential Equations
Maximilian Mauel, Johannes R. Hübers, David Berghaus +2
Ordinary differential equations (ODEs) are central to scientific modelling, but inferring their vector fields from noisy trajectories remains challenging. Current approaches such a…
In-Context Learning of Temporal Point Processes with Foundation Inference Models
David Berghaus, Patrick Seifner, Kostadin Cvejoski +2
Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamical rules and predict future even…
In-Context Learning of Stochastic Differential Equations with Foundation Inference Models
Patrick Seifner, Kostadin Cvejoski, David Berghaus +2
Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations, dictated by…
EVIL: Evolving Interpretable Algorithms for Zero-Shot Inference on Event Sequences and Time Series with LLMs
David Berghaus
We introduce EVIL (\textbf{EV}olving \textbf{I}nterpretable algorithms with \textbf{L}LMs), an approach that uses LLM-guided evolutionary search to discover simple, interpretable a…
Adaptive Loops and Memory in Transformers: Think Harder or Know More?
Markus Frey, Behzad Shomali, Ali Hamza Bashir +3
Chain-of-thought (CoT) prompting enables reasoning in language models but requires explicit verbalization of intermediate steps. Looped transformers offer an alternative by iterati…