7.8k citations · 8.4k across the 22 of their papers we have counts for
22 papers
Relational reasoning and inductive bias in transformers and large language models
Jesse Geerts, Andrew Liu, Stephanie Chan +2
Transformer-based models have demonstrated remarkable reasoning abilities, but the mechanisms underlying relational reasoning remain poorly understood. We investigate how transform…
Neural mechanisms of predictive processing: a collaborative community experiment through the OpenScope program
Ido Aizenbud, Nicholas Audette, Ryszard Auksztulewicz +50
This review synthesizes advances in predictive processing within the sensory cortex. Predictive processing theorizes that the brain continuously predicts sensory inputs, refining n…
Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
Anthony Zador, Sean Escola, Blake Richards +24
Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in…
When Does Re-initialization Work?
Sheheryar Zaidi, Tudor Berariu, Hyunjik Kim +4
Re-initializing a neural network during training has been observed to improve generalization in recent works. Yet it is neither widely adopted in deep learning practice nor is it o…
Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation
Sebastian Lee, Stefano Sarao Mannelli, Claudia Clopath +2
Continual learning - learning new tasks in sequence while maintaining performance on old tasks - remains particularly challenging for artificial neural networks. Surprisingly, the…
A study on the plasticity of neural networks
Tudor Berariu, Wojciech Czarnecki, Soham De +4
One aim shared by multiple settings, such as continual learning or transfer learning, is to leverage previously acquired knowledge to converge faster on the current task. Usually t…