5 papers · 1 filter
No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation
Bradley Stanley-Clamp, Anson Lei, Hannah M. Christensen +1
Climate emulation is an out-of-distribution (OOD) projection task. This is precisely the challenge where modern Machine Learning (ML) methods are most prone to failure. Consequentl…
Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention
Markus W. Baumgartner, Anson Lei, Joe Watson +1
Parametric system identification methods estimate the parameters of explicitly defined physical systems from data. Yet, they remain constrained by the need to provide an explicit f…
Intrinsically Interpretable Attention via Sparse Post-Training
Florent Draye, Anson Lei, Hsiao-Ru Pan +2
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-…
SPARTAN: A Sparse Transformer World Model Attending to What Matters
Anson Lei, Bernhard Schölkopf, Ingmar Posner
Capturing the interactions between entities in a structured way plays a central role in world models that flexibly adapt to changes in the environment. Recent works motivate the be…
Compete and Compose: Learning Independent Mechanisms for Modular World Models
Anson Lei, Frederik Nolte, Bernhard Schölkopf +1
We present COmpetitive Mechanisms for Efficient Transfer (COMET), a modular world model which leverages reusable, independent mechanisms across different environments. COMET is tra…