4 papers
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat +6
While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and…
Logic-Oriented Retriever Enhancement via Contrastive Learning
Wenxuan Zhang, Yuan-Hao Jiang, Changyong Qi +2
Large language models (LLMs) struggle in knowledge-intensive tasks, as retrievers often overfit to surface similarity and fail on queries involving complex logical relations. The c…
Towards the Training of Deeper Predictive Coding Neural Networks
Chang Qi, Matteo Forasassi, Thomas Lukasiewicz +1
Predictive coding networks are neural models that perform inference through an iterative energy minimization process, whose operations are local in space and time. While effective…
Benchmarking Predictive Coding Networks -- Made Simple
Luca Pinchetti, Chang Qi, Oleh Lokshyn +9
In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we propose a library, called PCX, that focus…