92 citations · 102 across the 7 of their papers we have counts for
8 papers
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat +6
While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and…
A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks
Tommaso Salvatori, Yuhang Song, Yordan Yordanov +6
Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. Training such models, however, is quite inefficient and unstabl…
Predictive Coding beyond Gaussian Distributions
Luca Pinchetti, Tommaso Salvatori, Yordan Yordanov +3
A large amount of recent research has the far-reaching goal of finding training methods for deep neural networks that can serve as alternatives to backpropagation (BP). A prominent…
Bird-Eye Transformers for Text Generation Models
Lei Sha, Yuhang Song, Yordan Yordanov +2
Transformers have become an indispensable module for text generation models since their great success in machine translation. Previous works attribute the~success of transformers t…
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup
Yordan Yordanov, Vid Kocijan, Thomas Lukasiewicz +1
Training a model to provide natural language explanations (NLEs) for its predictions usually requires the acquisition of task-specific NLEs, which is time- and resource-consuming.…
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution
Yordan Yordanov, Oana-Maria Camburu, Vid Kocijan +1
Hard cases of pronoun resolution have been used as a long-standing benchmark for commonsense reasoning. In the recent literature, pre-trained language models have been used to obta…
WikiCREM: A Large Unsupervised Corpus for Coreference Resolution
Vid Kocijan, Oana-Maria Camburu, Ana-Maria Cretu +3
Pronoun resolution is a major area of natural language understanding. However, large-scale training sets are still scarce, since manually labelling data is costly. In this work, we…
A Surprisingly Robust Trick for Winograd Schema Challenge
Vid Kocijan, Ana-Maria Cretu, Oana-Maria Camburu +2
The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. In this p…