14 papers
A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Jiajun Bao, Zihao Qi, Toni J. B. Liu +6
Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identifie…
Deep Embedded Multiplicative DMD for Algebra-Preserving Koopman Learning
Kelan Gray, Finlay Brown, Nicolas Boullé +1
Koopman theory turns nonlinear dynamics into a linear spectral problem. In computation, however, everything depends on a hard finite-dimensional choice: the observables must be exp…
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Oussama Zekri, Nicolas Boullé
Discrete diffusion models have recently gained significant attention due to their ability to process complex discrete structures for language modeling. However, fine-tuning these m…
Multiplicative Dynamic Mode Decomposition
Nicolas Boullé, Matthew J. Colbrook
Koopman operators are infinite-dimensional operators that linearize nonlinear dynamical systems, facilitating the study of their spectral properties and enabling the prediction of…
What's in a prompt? Language models encode literary style in prompt embeddings
Raphaël Sarfati, Haley Moller, Toni J. B. Liu +2
Large language models use high-dimensional latent spaces to encode and process textual information. Much work has investigated how the conceptual content of words translates into g…
Density estimation with LLMs: a geometric investigation of in-context learning trajectories
Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati +1
Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time series forecasting. This work investigate…