1 citations · 1 across the 4 of their papers we have counts for
6 papers
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat +6
While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and…
Predictive Coding beyond Gaussian Distributions
Luca Pinchetti, Tommaso Salvatori, Yordan Yordanov +3
A large amount of recent research has the far-reaching goal of finding training methods for deep neural networks that can serve as alternatives to backpropagation (BP). A prominent…
Bird-Eye Transformers for Text Generation Models
Lei Sha, Yuhang Song, Yordan Yordanov +2
Transformers have become an indispensable module for text generation models since their great success in machine translation. Previous works attribute the~success of transformers t…
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution
Yordan Yordanov, Oana-Maria Camburu, Vid Kocijan +1
Hard cases of pronoun resolution have been used as a long-standing benchmark for commonsense reasoning. In the recent literature, pre-trained language models have been used to obta…
WikiCREM: A Large Unsupervised Corpus for Coreference Resolution
Vid Kocijan, Oana-Maria Camburu, Ana-Maria Cretu +3
Pronoun resolution is a major area of natural language understanding. However, large-scale training sets are still scarce, since manually labelling data is costly. In this work, we…
A Surprisingly Robust Trick for Winograd Schema Challenge
Vid Kocijan, Ana-Maria Cretu, Oana-Maria Camburu +2
The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. In this p…