302 citations · 444 across the 25 of their papers we have counts for
26 papers
Tiny Recursive Message Passing
Tommaso Salvatori, Matteo Forasassi, Alexis Toumi +1
Message passing neural networks have shown remarkable performance in algorithmic and constraint reasoning, due to their ability to encode the structure of the task directly in the…
Bio-inspired Learning and Decision-Making with Probabilistic In-Memory Computing Hardware: Part 2
Thomas Dalgaty, Eiji Kawasaki, Miguel de Prado +3
This report extends our previous work (Part 1), which introduced an energy-based model for learning and decision-making under uncertainty. The model leverages stochastic Langevin d…
Bio-inspired Learning and Decision-Making with Probabilistic In-Memory Computing Hardware: Part 1
Thomas Dalgaty, Eiji Kawasaki, Miguel de Prado +2
Learning and decision-making in animals are often modeled as Bayesian processes, where sensory evidence is integrated with prior beliefs to guide behavior in the face of uncertaint…
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat +6
While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and…
Faster Predictive Coding Networks via Better Initialization
Luca Pinchetti, Simon Frieder, Thomas Lukasiewicz +1
Research aimed at scaling up neuroscience inspired learning algorithms for neural networks is accelerating. Recently, a key research area has been the study of energy-based learnin…
Towards the Training of Deeper Predictive Coding Neural Networks
Chang Qi, Matteo Forasassi, Thomas Lukasiewicz +1
Predictive coding networks are neural models that perform inference through an iterative energy minimization process, whose operations are local in space and time. While effective…
Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks
Ankur Mali, Tommaso Salvatori, Alexander Ororbia
Energy-based learning algorithms, such as predictive coding (PC), have garnered significant attention in the machine learning community due to their theoretical properties, such as…
Benchmarking Predictive Coding Networks -- Made Simple
Luca Pinchetti, Chang Qi, Oleh Lokshyn +9
In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we propose a library, called PCX, that focus…
A Review of Neuroscience-Inspired Machine Learning
Alexander Ororbia, Ankur Mali, Adam Kohan +2
One major criticism of deep learning centers around the biological implausibility of the credit assignment schema used for learning -- backpropagation of errors. This implausibilit…
Associative Memories in the Feature Space
Tommaso Salvatori, Beren Millidge, Yuhang Song +2
An autoassociative memory model is a function that, given a set of data points, takes as input an arbitrary vector and outputs the most similar data point from the memorized set. H…
Brain-inspired Computational Intelligence via Predictive Coding
Tommaso Salvatori, Ankur Mali, Christopher L. Buckley +4
Artificial intelligence (AI) is rapidly becoming one of the key technologies of this century. The majority of results in AI thus far have been achieved using deep neural networks t…
Predictive Coding beyond Correlations
Tommaso Salvatori, Luca Pinchetti, Amine M'Charrak +2
Recently, there has been extensive research on the capabilities of biologically plausible algorithms. In this work, we show how one of such algorithms, called predictive coding, is…
Mathematical Capabilities of ChatGPT
Simon Frieder, Luca Pinchetti, Alexis Chevalier +5
We investigate the mathematical capabilities of two iterations of ChatGPT (released 9-January-2023 and 30-January-2023) and of GPT-4 by testing them on publicly available datasets,…
Robust Graph Representation Learning via Predictive Coding
Billy Byiringiro, Tommaso Salvatori, Thomas Lukasiewicz
Predictive coding is a message-passing framework initially developed to model information processing in the brain, and now also topic of research in machine learning due to some in…
A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks
Tommaso Salvatori, Yuhang Song, Yordan Yordanov +6
Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. Training such models, however, is quite inefficient and unstabl…
Predictive Coding beyond Gaussian Distributions
Luca Pinchetti, Tommaso Salvatori, Yordan Yordanov +3
A large amount of recent research has the far-reaching goal of finding training methods for deep neural networks that can serve as alternatives to backpropagation (BP). A prominent…
Bird-Eye Transformers for Text Generation Models
Lei Sha, Yuhang Song, Yordan Yordanov +2
Transformers have become an indispensable module for text generation models since their great success in machine translation. Previous works attribute the~success of transformers t…
A Theoretical Framework for Inference and Learning in Predictive Coding Networks
Beren Millidge, Yuhang Song, Tommaso Salvatori +2
Predictive coding (PC) is an influential theory in computational neuroscience, which argues that the cortex forms unsupervised world models by implementing a hierarchical process o…
Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning
Beren Millidge, Yuhang Song, Tommaso Salvatori +2
How the brain performs credit assignment is a fundamental unsolved problem in neuroscience. Many `biologically plausible' algorithms have been proposed, which compute gradients tha…
Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Beren Millidge, Tommaso Salvatori, Yuhang Song +2
The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning. However, it requires sequential backward updates a…
Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models
Beren Millidge, Tommaso Salvatori, Yuhang Song +2
A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories…
Learning on Arbitrary Graph Topologies via Predictive Coding
Tommaso Salvatori, Luca Pinchetti, Beren Millidge +4
Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps a data point to its prediction, and a backward pass that propagate…
Associative Memories via Predictive Coding
Tommaso Salvatori, Yuhang Song, Yujian Hong +5
Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in…
Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
Tommaso Salvatori, Yuhang Song, Thomas Lukasiewicz +2
Predictive coding networks (PCNs) are an influential model for information processing in the brain. They have appealing theoretical interpretations and offer a single mechanism tha…