6 citations · 6 across the 7 of their papers we have counts for
6 papers · 1 filter
The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priors
Toni J. B. Liu, Jiajun Bao, Yizhou Liu +4
What does a language model predict when it has few clues? The answer lurks in its unembedding geometry: a single direction of the unembedding matrix encodes the unigram distributio…
A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Jiajun Bao, Zihao Qi, Toni J. B. Liu +4
Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identifie…
Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanism
Jiajun Bao, Nicolas Boullé, Toni J. B. Liu +2
Large language models (LLMs) have demonstrated emergent in-context learning (ICL) capabilities across a range of tasks, including zero-shot time-series forecasting. We show that te…
Density estimation with LLMs: a geometric investigation of in-context learning trajectories
Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati +1
Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time series forecasting. This work investigate…
Lines of Thought in Large Language Models
Raphaël Sarfati, Toni J. B. Liu, Nicolas Boullé +1
Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transf…
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law
Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati +1
Pretrained large language models (LLMs) are surprisingly effective at performing zero-shot tasks, including time-series forecasting. However, understanding the mechanisms behind su…