6 papers
Defining and Generating Compositional Structure
Eric Elmoznino, Tejas Kasetty, Blaise Agüera y Arcas +2
Compositional structure is pervasive in nature and across scales: atoms combine into molecules, molecules into cells, and cells into organisms, while further compositional hierarch…
Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Yinhan Lu, Eric Elmoznino, Léo Gagnon +3
Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood com…
Causal Network Discovery from Interventional Count Data with Latent Linear DAGs
Yijiao Zhang, Hongzhe Li
The increasing availability of interventional data offers new opportunities for causal discovery, with gene perturbation studies providing a prominent example. Such data are typica…
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Leo Gagnon, Eric Elmoznino, Sarthak Mittal +4
The rapid adaptation ability of auto-regressive foundation models is often attributed to the diversity of their pre-training data. This is because, from a Bayesian standpoint, mini…
In-context learning and Occam's razor
Eric Elmoznino, Tom Marty, Tejas Kasetty +5
A central goal of machine learning is generalization. While the No Free Lunch Theorem states that we cannot obtain theoretical guarantees for generalization without further assumpt…
Evaluating Interventional Reasoning Capabilities of Large Language Models
Tejas Kasetty, Divyat Mahajan, Gintare Karolina Dziugaite +2
Numerous decision-making tasks require estimating causal effects under interventions on different parts of a system. As practitioners consider using large language models (LLMs) to…