79 citations · 172 across the 20 of their papers we have counts for
22 papers
The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priors
Toni J. B. Liu, Jiajun Bao, Yizhou Liu +4
What does a language model predict when it has few clues? The answer lurks in its unembedding geometry: a single direction of the unembedding matrix encodes the unigram distributio…
A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Jiajun Bao, Zihao Qi, Toni J. B. Liu +4
Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identifie…
Deep Embedded Multiplicative DMD for Algebra-Preserving Koopman Learning
Kelan Gray, Finlay Brown, Nicolas Boullé +1
Koopman theory turns nonlinear dynamics into a linear spectral problem. In computation, however, everything depends on a hard finite-dimensional choice: the observables must be exp…
What's in a prompt? Language models encode literary style in prompt embeddings
Raphaël Sarfati, Haley Moller, Toni J. B. Liu +2
Large language models use high-dimensional latent spaces to encode and process textual information. Much work has investigated how the conceptual content of words translates into g…
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Oussama Zekri, Nicolas Boullé
Discrete diffusion models have recently gained significant attention due to their ability to process complex discrete structures for language modeling. However, fine-tuning these m…
Operator learning regularization for macroscopic permeability prediction in dual-scale flow problem
Christina Runkel, Sinan Xiao, Nicolas Boullé +1
Liquid composites moulding is an important manufacturing technology for fibre reinforced composites, due to its cost-effectiveness. Challenges lie in the optimisation of the proces…
Density estimation with LLMs: a geometric investigation of in-context learning trajectories
Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati +1
Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time series forecasting. This work investigate…
Large Language Models as Markov Chains
Oussama Zekri, Ambroise Odonnat, Abdelhakim Benechehab +3
Large language models (LLMs) are remarkably efficient across a wide range of natural language processing tasks and well beyond them. However, a comprehensive theoretical analysis o…
Lines of Thought in Large Language Models
Raphaël Sarfati, Toni J. B. Liu, Nicolas Boullé +1
Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transf…
Structure-Preserving Operator Learning
Nacime Bouziani, Nicolas Boullé
Learning complex dynamics driven by partial differential equations directly from data holds great promise for fast and accurate simulations of complex physical systems. In most cas…
Multiplicative Dynamic Mode Decomposition
Nicolas Boullé, Matthew J. Colbrook
Koopman operators are infinite-dimensional operators that linearize nonlinear dynamical systems, facilitating the study of their spectral properties and enabling the prediction of…
Randomized Nyström approximation of non-negative self-adjoint operators
David Persson, Nicolas Boullé, Daniel Kressner
The randomized singular value decomposition (SVD) has become a popular approach to computing cheap, yet accurate, low-rank approximations to matrices due to its efficiency and stro…
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law
Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati +1
Pretrained large language models (LLMs) are surprisingly effective at performing zero-shot tasks, including time-series forecasting. However, understanding the mechanisms behind su…
Operator learning without the adjoint
Nicolas Boullé, Diana Halikias, Samuel E. Otto +1
There is a mystery at the heart of operator learning: how can one recover a non-self-adjoint operator from data without probing the adjoint? Current practical approaches suggest th…
On the Convergence of Hermitian Dynamic Mode Decomposition
Nicolas Boullé, Matthew J. Colbrook
We study the convergence of Hermitian Dynamic Mode Decomposition (DMD) to the spectral properties of self-adjoint Koopman operators. Hermitian DMD is a data-driven method that appr…
A Mathematical Guide to Operator Learning
Nicolas Boullé, Alex Townsend
Operator learning aims to discover properties of an underlying dynamical system or partial differential equation (PDE) from data. Here, we present a step-by-step guide to operator…
Multivariate rational approximation of functions with curves of singularities
Nicolas Boullé, Astrid Herremans, Daan Huybrechs
Functions with singularities are notoriously difficult to approximate with conventional approximation schemes. In computational applications, they are often resolved with low-order…
Principled interpolation of Green's functions learned from data
Harshwardhan Praveen, Nicolas Boulle, Christopher Earls
We present a data-driven approach to mathematically model physical systems whose governing partial differential equations are unknown, by learning their associated Green's function…
Two-Component 3D Atomic Bose-Einstein Condensates Support Complex Stable Patterns
N. Boullé, I. Newell, P. E. Farrell +1
We report the computational discovery of complex, topologically charged, and spectrally stable states in three-dimensional multi-component nonlinear wave systems of nonlinear Schr{…
Deflation-based Identification of Nonlinear Excitations of the 3D Gross--Pitaevskii equation
N. Boullé, E. G. Charalampidis, P. E. Farrell +1
We present previously unknown solutions to the 3D Gross--Pitaevskii equation describing atomic Bose-Einstein condensates. This model supports elaborate patterns, including excited…
Bifurcation analysis of stationary solutions of two-dimensional coupled Gross-Pitaevskii equations using deflated continuation
E. G Charalampidis, N. Boullé, P. E. Farrell +1
Recently, a novel bifurcation technique known as the deflated continuation method (DCM) was applied to the single-component nonlinear Schrödinger (NLS) equation with a parabolic tr…
Classification of chaotic time series with deep learning
Nicolas Boullé, Vassilios Dallas, Yuji Nakatsukasa +1
We use standard deep neural networks to classify univariate time series generated by discrete and continuous dynamical systems based on their chaotic or non-chaotic behaviour. Our…