9 citations · 12 across the 6 of their papers we have counts for
7 papers · 1 filter
The Validity of Coreference-based Evaluations of Natural Language Understanding
Ian Porada
In this thesis, I refine our understanding as to what conclusions we can reach from coreference-based evaluations by expanding existing evaluation practices and considering the ext…
Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution
Ian Porada, Jackie Chi Kit Cheung
Challenge sets such as the Winograd Schema Challenge (WSC) are used to benchmark systems' ability to resolve ambiguities in natural language. If one assumes as in existing work tha…
A Controlled Reevaluation of Coreference Resolution Models
Ian Porada, Xiyuan Zou, Jackie Chi Kit Cheung
All state-of-the-art coreference resolution (CR) models involve finetuning a pretrained language model. Whether the superior performance of one CR model over another is due to the…
Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective
Ian Porada, Alexandra Olteanu, Kaheer Suleman +2
It is increasingly common to evaluate the same coreference resolution (CR) model on multiple datasets. Do these multi-dataset evaluations allow us to draw meaningful conclusions ab…
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge
Ian Porada, Alessandro Sordoni, Jackie Chi Kit Cheung
Transformer models pre-trained with a masked-language-modeling objective (e.g., BERT) encode commonsense knowledge as evidenced by behavioral probes; however, the extent to which t…
Modeling Event Plausibility with Consistent Conceptual Abstraction
Ian Porada, Kaheer Suleman, Adam Trischler +1
Understanding natural language requires common sense, one aspect of which is the ability to discern the plausibility of events. While distributional models -- most recently pre-tra…