1 citations · 1 across the 6 of their papers we have counts for
8 papers
Gated Target Propagation for Compositional Generalization in Continual Learning
Abdel Mfougouon Njupoun, Colin Bredenberg, Blake Aaron Richards +1
Continual learning is typically framed as acquiring new knowledge without catastrophically forgetting previous tasks. However, a flexible continual learner should also be able to r…
A game theory for foundation models shows new paths to rational cooperation through similarity inference
Alexander Meulemans, Maciej Wołczyk, Marissa A. Weis +11
As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles governing their collective behavior is…
Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
Ezekiel Williams, Alexandre Payeur, Guillaume Lajoie
Biological and neuromorphic recurrent neural networks (RNNs) are subject to spatial and temporal locality constraints on the information that can plausibly be used during learning.…
Beyond Distribution Sharpening: The Importance of Task Rewards
Sarthak Mittal, Leo Gagnon, Guillaume Lajoie
Frontier models have demonstrated exceptional capabilities following the integration of task-reward-based reinforcement learning (RL) into their training pipelines, enabling system…
Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning
Seijin Kobayashi, Yanick Schimpf, Maximilian Schlegel +12
Large-scale autoregressive models pretrained on next-token prediction and finetuned with reinforcement learning (RL) have achieved unprecedented success on many problem domains. Du…
Embedded Universal Predictive Intelligence: a coherent framework for multi-agent learning
Alexander Meulemans, Rajai Nasser, Maciej Wołczyk +10
The standard theory of model-free reinforcement learning assumes that the environment dynamics are stationary and that agents are decoupled from their environment, such that polici…
Tracing the Representation Geometry of Language Models from Pretraining to Post-training
Melody Zixuan Li, Kumar Krishna Agrawal, Arna Ghosh +4
Standard training metrics like loss fail to explain the emergence of complex capabilities in large language models. We take a spectral approach to investigate the geometry of learn…
Sufficient conditions for offline reactivation in recurrent neural networks
Nanda H. Krishna, Colin Bredenberg, Daniel Levenstein +2
During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagement. However, the precise conditions under…