4 papers
The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color
Cory Paik, Stéphane Aroca-Ouellette, Alessandro Roncone +1
Recent work has raised concerns about the inherent limitations of text-only pretraining. In this paper, we first demonstrate that reporting bias, the tendency of people to not stat…
PROST: Physical Reasoning of Objects through Space and Time
Stéphane Aroca-Ouellette, Cory Paik, Alessandro Roncone +1
We present a new probing dataset named PROST: Physical Reasoning about Objects Through Space and Time. This dataset contains 18,736 multiple-choice questions made from 14 manually…
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data
Demetres Kostas, Stephane Aroca-Ouellette, Frank Rudzicz
Deep neural networks (DNNs) used for brain-computer-interface (BCI) classification are commonly expected to learn general features when trained across a variety of contexts, such t…
On Losses for Modern Language Models
Stephane Aroca-Ouellette, Frank Rudzicz
BERT set many state-of-the-art results over varied NLU benchmarks by pre-training over two tasks: masked language modelling (MLM) and next sentence prediction (NSP), the latter of…