7 papers
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat +6
While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and…
Faster Predictive Coding Networks via Better Initialization
Luca Pinchetti, Simon Frieder, Thomas Lukasiewicz +1
Research aimed at scaling up neuroscience inspired learning algorithms for neural networks is accelerating. Recently, a key research area has been the study of energy-based learnin…
Brain-inspired Computational Intelligence via Predictive Coding
Tommaso Salvatori, Ankur Mali, Christopher L. Buckley +4
Artificial intelligence (AI) is rapidly becoming one of the key technologies of this century. The majority of results in AI thus far have been achieved using deep neural networks t…
Towards the Training of Deeper Predictive Coding Neural Networks
Chang Qi, Matteo Forasassi, Thomas Lukasiewicz +1
Predictive coding networks are neural models that perform inference through an iterative energy minimization process, whose operations are local in space and time. While effective…
Benchmarking Predictive Coding Networks -- Made Simple
Luca Pinchetti, Chang Qi, Oleh Lokshyn +9
In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we propose a library, called PCX, that focus…
Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks
Ankur Mali, Tommaso Salvatori, Alexander Ororbia
Energy-based learning algorithms, such as predictive coding (PC), have garnered significant attention in the machine learning community due to their theoretical properties, such as…