52 citations · 56 across the 3 of their papers we have counts for
6 papers
Automatic Document Selection for Efficient Encoder Pretraining
Yukun Feng, Patrick Xia, Benjamin Van Durme +1
Building pretrained language models is considered expensive and data-intensive, but must we increase dataset size to achieve better performance? We propose an alternative to larger…
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal +53
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of auto…
SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training
Huda Khayrallah, João Sedoc
Non-task-oriented dialog models suffer from poor quality and non-diverse responses. To overcome limited conversational data, we apply Simulated Multiple Reference Training (SMRT; K…
COD3S: Diverse Generation with Discrete Semantic Signatures
Nathaniel Weir, João Sedoc, Benjamin Van Durme
We present COD3S, a novel method for generating semantically diverse sentences using neural sequence-to-sequence (seq2seq) models. Conditioned on an input, seq2seq models typically…
Measuring the `I don't know' Problem through the Lens of Gricean Quantity
Huda Khayrallah, João Sedoc
We consider the intrinsic evaluation of neural generative dialog models through the lens of Grice's Maxims of Conversation (1975). Based on the maxim of Quantity (be informative),…
Incremental Neural Coreference Resolution in Constant Memory
Patrick Xia, João Sedoc, Benjamin Van Durme
We investigate modeling coreference resolution under a fixed memory constraint by extending an incremental clustering algorithm to utilize contextualized encoders and neural compon…