193 citations · 199 across the 13 of their papers we have counts for
5 papers · 1 filter
Can In-Context Learning Support Intrinsic Curiosity?
Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald +5
Effective machine learning depends not only on how we model data, but also on what data we choose to collect. While large sequence models have revolutionized data modeling, the pro…
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Leo Gagnon, Eric Elmoznino, Sarthak Mittal +4
The rapid adaptation ability of auto-regressive foundation models is often attributed to the diversity of their pre-training data. This is because, from a Bayesian standpoint, mini…
In-context learning and Occam's razor
Eric Elmoznino, Tom Marty, Tejas Kasetty +5
A central goal of machine learning is generalization. While the No Free Lunch Theorem states that we cannot obtain theoretical guarantees for generalization without further assumpt…
Does learning the right latent variables necessarily improve in-context learning?
Sarthak Mittal, Eric Elmoznino, Leo Gagnon +4
Large autoregressive models like Transformers can solve tasks through in-context learning (ICL) without learning new weights, suggesting avenues for efficiently solving new tasks.…
Amortizing intractable inference in large language models
Edward J. Hu, Moksh Jain, Eric Elmoznino +4
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits tractable querying of this knowled…