Publications (104)
Learning Neural Causal Models from Unknown Interventions
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal +6
State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations
Alex Lamb, Jonathan Binas, Anirudh Goyal +5
Can AI Be as Creative as Humans?
Haonan Wang, James Zou, Michael Mozer +8
Discrete Key-Value Bottleneck
Frederik Träuble, Anirudh Goyal, Nasim Rahaman +4
On Disentangled Representations Learned From Correlated Data
Frederik Träuble, Elliot Creager, Niki Kilbertus +5
$α$-TCVAE: On the relationship between Disentanglement and Diversity
Cristian Meo, Louis Mahon, Anirudh Goyal +1
Fast and Slow Learning of Recurrent Independent Mechanisms
Kanika Madan, Nan Rosemary Ke, Anirudh Goyal +2
Scaling Test-Time Compute for Agentic Coding
Joongwon Kim, Wannan Yang, Kelvin Niu +13
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
Ossama Ahmed, Frederik Träuble, Anirudh Goyal +5
Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling
Yiran Zhao, Wenyue Zheng, Tianle Cai +4
Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
Alex Lamb, Jonathan Binas, Anirudh Goyal +4
Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems
Anirudh Goyal, Alex Lamb, Phanideep Gampa +5
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
Yoshua Bengio, Tristan Deleu, Nasim Rahaman +5
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli +1
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks
Nan Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk +4
On the Impossibility of Retrain Equivalence in Machine Unlearning
Jiatong Yu, Yinghui He, Anirudh Goyal +1
Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discovery
Tingting Chen, Beibei Lin, Srinivas Anumasa +5
Masked Generative Priors Improve World Models Sequence Modelling Capabilities
Cristian Meo, Mircea Lica, Zarif Ikram +6
Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning
Riashat Islam, Hongyu Zang, Anirudh Goyal +6
Learning to Induce Causal Structure
Nan Rosemary Ke, Silvia Chiappa, Jane Wang +7
A Theory for Emergence of Complex Skills in Language Models
Sanjeev Arora, Anirudh Goyal
Inductive Biases for Deep Learning of Higher-Level Cognition
Anirudh Goyal, Yoshua Bengio
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement
Yuxi Xie, Anirudh Goyal, Xiaobao Wu +5
S2RMs: Spatially Structured Recurrent Modules
Nasim Rahaman, Anirudh Goyal, Muhammad Waleed Gondal +5
Reasoning Robustness of LLMs to Adversarial Typographical Errors
Esther Gan, Yiran Zhao, Liying Cheng +5
Cycle Consistency Driven Object Discovery
Aniket Didolkar, Anirudh Goyal, Yoshua Bengio
Small-GAN: Speeding Up GAN Training Using Core-sets
Samarth Sinha, Han Zhang, Anirudh Goyal +3
Learning Neural Causal Models with Active Interventions
Nino Scherrer, Olexa Bilaniuk, Yashas Annadani +7
Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior
Ayush Chakravarthy, Trang Nguyen, Anirudh Goyal +2
Escaping the Cognitive Well: Efficient Competition Math with Off-the-Shelf Models
Xingyu Dang, Rohit Agarwal, Rodrigo Porto +3
Stories in the Eye: Contextual Visual Interactions for Efficient Video to Language Translation
Anirudh Goyal, Marius Leordeanu
Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning
Simon Frieder, Jonas Bayer, Sam Looi +13
Physical Reasoning and Object Planning for Household Embodied Agents
Ayush Agrawal, Raghav Prabhakar, Anirudh Goyal +1
Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision
Dulhan Jayalath, Shashwat Goel, Thomas Foster +5
Aligning Text-to-Image Diffusion Models with Reward Backpropagation
Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak +1
Unnatural Languages Are Not Bugs but Features for LLMs
Keyu Duan, Yiran Zhao, Zhili Feng +9
InfoBot: Transfer and Exploration via the Information Bottleneck
Anirudh Goyal, Riashat Islam, Daniel Strouse +5
Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning
Simran Kaur, Simon Park, Anirudh Goyal +1
Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
Alex Lamb, Anirudh Goyal, Agnieszka SÅowik +3
Can Models Learn Skill Composition from Examples?
Haoyu Zhao, Simran Kaur, Dingli Yu +2
Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel
Dianbo Liu, Vedant Shah, Oussama Boussif +6
Discrete-Valued Neural Communication
Dianbo Liu, Alex Lamb, Kenji Kawaguchi +4
Diversity inducing Information Bottleneck in Model Ensembles
Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal +3
Representation Learning in Deep RL via Discrete Information Bottleneck
Riashat Islam, Hongyu Zang, Manan Tomar +8
Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples
Samarth Sinha, Zhengli Zhao, Anirudh Goyal +2
Leveraging the Third Dimension in Contrastive Learning
Sumukh Aithal, Anirudh Goyal, Alex Lamb +2
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Rohan Anil, Sebastian Borgeaud +1340
Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases
Cristian Meo, Akihiro Nakano, Mircea LicÄ +7
The Alien Space of Science: Sampling Coherent but Cognitively Unavailable Research Directions
Alejandro H. Artiles, Martin Weiss, Levin Brinkmann +6
Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
Siyuan Guo, Aniket Didolkar, Nan Rosemary Ke +3
Recurrent Independent Mechanisms
Anirudh Goyal, Alex Lamb, Jordan Hoffmann +4
Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
Yuxi Xie, Anirudh Goyal, Wenyue Zheng +4
Generalization of Equilibrium Propagation to Vector Field Dynamics
Benjamin Scellier, Anirudh Goyal, Jonathan Binas +2
ACtuAL: Actor-Critic Under Adversarial Learning
Anirudh Goyal, Nan Rosemary Ke, Alex Lamb +4
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté +2
Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors
Aniket Didolkar, Nicolas Ballas, Sanjeev Arora +1
Recall Traces: Backtracking Models for Efficient Reinforcement Learning
Anirudh Goyal, Philemon Brakel, William Fedus +5
HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds
Tingting Chen, Beibei Lin, Zifeng Yuan +5
Test-time Adaptation with Slot-Centric Models
Mihir Prabhudesai, Anirudh Goyal, Sujoy Paul +6
Experience Graphs: The Data Foundation for Self-Improving Agents
Gang Liao, Yujia He, Abdullah Ozturk +22
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates
Cristian Meo, Ksenia Sycheva, Anirudh Goyal +1
Zero-Shot Object-Centric Representation Learning
Aniket Didolkar, Andrii Zadaianchuk, Anirudh Goyal +4
Coordination Among Neural Modules Through a Shared Global Workspace
Anirudh Goyal, Aniket Didolkar, Alex Lamb +7
An Actor-Critic Algorithm for Sequence Prediction
Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu +5
ASTRO: Teaching Language Models to Reason by Reflecting and Backtracking In-Context
Joongwon Kim, Anirudh Goyal, Liang Tan +3
AI-Assisted Generation of Difficult Math Questions
Vedant Shah, Dingli Yu, Kaifeng Lyu +8
Narrowing the Focus: Learned Optimizers for Pretrained Models
Gus Kristiansen, Mark Sandler, Andrey Zhmoginov +4
Retrieval-Augmented Reinforcement Learning
Anirudh Goyal, Abram L. Friesen, Andrea Banino +13
Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models
Dingli Yu, Simran Kaur, Arushi Gupta +3
FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
Qiran Zou, Hou Hei Lam, Wenhao Zhao +11
On the Generalization and Adaption Performance of Causal Models
Nino Scherrer, Anirudh Goyal, Stefan Bauer +2
Uniform Priors for Data-Efficient Transfer
Samarth Sinha, Karsten Roth, Anirudh Goyal +3
A Systematic Examination of Preference Learning through the Lens of Instruction-Following
Joongwon Kim, Anirudh Goyal, Aston Zhang +6
Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning
Dianbo Liu, Vedant Shah, Oussama Boussif +6
Neural Production Systems: Learning Rule-Governed Visual Dynamics
Anirudh Goyal, Aniket Didolkar, Nan Rosemary Ke +5
Learning Powerful Policies by Using Consistent Dynamics Model
Shagun Sodhani, Anirudh Goyal, Tristan Deleu +3
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu, Moksh Jain, Bonaventure Dossou +10
Untangling tradeoffs between recurrence and self-attention in neural networks
Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal +3
Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?
Simon Park, Abhishek Panigrahi, Yun Cheng +3
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
Kaifeng Lyu, Haoyu Zhao, Xinran Gu +3
Variational Causal Networks: Approximate Bayesian Inference over Causal Structures
Yashas Annadani, Jonas Rothfuss, Alexandre Lacoste +4
Rethinking Thinking Tokens: LLMs as Improvement Operators
Lovish Madaan, Aniket Didolkar, Suchin Gururangan +6
Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules
Sarthak Mittal, Alex Lamb, Anirudh Goyal +5
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding
Nan Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk +4
Towards Causal Representation Learning
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer +4
DiscoGen: Learning to Discover Gene Regulatory Networks
Nan Rosemary Ke, Sara-Jane Dunn, Jorg Bornschein +11
Robust Representation Learning via Perceptual Similarity Metrics
Saeid Asgari Taghanaki, Kristy Choi, Amir Khasahmadi +1
Professor Forcing: A New Algorithm for Training Recurrent Networks
Alex Lamb, Anirudh Goyal, Ying Zhang +3
Real Robot Challenge: A Robotics Competition in the Cloud
Stefan Bauer, Felix Widmaier, Manuel Wüthrich +39
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
Antonis Antoniades, Albert Ãrwall, Kexun Zhang +3
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Nan Rosemary Ke, Aniket Didolkar, Sarthak Mittal +7
Unlearning via Sparse Representations
Vedant Shah, Frederik Träuble, Ashish Malik +5
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
David Krueger, Tegan Maharaj, János Kramár +7
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning
Aniket Didolkar, Kshitij Gupta, Anirudh Goyal +4
The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget
Anirudh Goyal, Yoshua Bengio, Matthew Botvinick +1
Leveraging Communication Topologies Between Learning Agents in Deep Reinforcement Learning
Dhaval Adjodah, Dan Calacci, Abhimanyu Dubey +4
The Llama 3 Herd of Models
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri +556
TC-VAE: Uncovering Out-of-Distribution Data Generative Factors
Cristian Meo, Anirudh Goyal, Justin Dauwels
Transformers with Competitive Ensembles of Independent Mechanisms
Alex Lamb, Di He, Anirudh Goyal +4
Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives
Anirudh Goyal, Shagun Sodhani, Jonathan Binas +3