193 citations · 199 across the 13 of their papers we have counts for
14 papers
Defining and Generating Compositional Structure
Eric Elmoznino, Tejas Kasetty, Blaise Agüera y Arcas +2
Compositional structure is pervasive in nature and across scales: atoms combine into molecules, molecules into cells, and cells into organisms, while further compositional hierarch…
Can In-Context Learning Support Intrinsic Curiosity?
Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald +5
Effective machine learning depends not only on how we model data, but also on what data we choose to collect. While large sequence models have revolutionized data modeling, the pro…
Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Yinhan Lu, Eric Elmoznino, Léo Gagnon +3
Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood com…
Causal Network Discovery from Interventional Count Data with Latent Linear DAGs
Yijiao Zhang, Hongzhe Li
The increasing availability of interventional data offers new opportunities for causal discovery, with gene perturbation studies providing a prominent example. Such data are typica…
Engineering Sentience
Konstantin Demin, Taylor Webb, Eric Elmoznino +1
We spell out a definition of sentience that may be useful for designing and building it in machines. We propose that for sentience to be meaningful for AI, it must be fleshed out i…
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Leo Gagnon, Eric Elmoznino, Sarthak Mittal +4
The rapid adaptation ability of auto-regressive foundation models is often attributed to the diversity of their pre-training data. This is because, from a Bayesian standpoint, mini…
Multi-agent cooperation through learning-aware policy gradients
Alexander Meulemans, Seijin Kobayashi, Johannes von Oswald +6
Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. How can we achieve cooperation among self-interested, independent lear…
A Complexity-Based Theory of Compositionality
Eric Elmoznino, Thomas Jiralerspong, Yoshua Bengio +1
Compositionality is believed to be fundamental to intelligence. In humans, it underlies the structure of thought, language, and higher-level reasoning. In AI, compositional represe…
In-context learning and Occam's razor
Eric Elmoznino, Tom Marty, Tejas Kasetty +5
A central goal of machine learning is generalization. While the No Free Lunch Theorem states that we cannot obtain theoretical guarantees for generalization without further assumpt…
Does learning the right latent variables necessarily improve in-context learning?
Sarthak Mittal, Eric Elmoznino, Leo Gagnon +4
Large autoregressive models like Transformers can solve tasks through in-context learning (ICL) without learning new weights, suggesting avenues for efficiently solving new tasks.…
Amortizing intractable inference in large language models
Edward J. Hu, Moksh Jain, Eric Elmoznino +4
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits tractable querying of this knowled…
Discrete, compositional, and symbolic representations through attractor dynamics
Andrew Nam, Eric Elmoznino, Nikolay Malkin +3
Symbolic systems are powerful frameworks for modeling cognitive processes as they encapsulate the rules and relationships fundamental to many aspects of human reasoning and behavio…
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
Patrick Butlin, Robert Long, Eric Elmoznino +16
Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and…
Sources of Richness and Ineffability for Phenomenally Conscious States
Xu Ji, Eric Elmoznino, George Deane +5
Conscious states (states that there is something it is like to be in) seem both rich or full of detail, and ineffable or hard to fully describe or recall. The problem of ineffabili…