52 citations · 64 across the 3 of their papers we have counts for
6 papers
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal +53
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of auto…
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
Moin Nadeem, Tianxing He, Kyunghyun Cho +1
This work studies the widely adopted ancestral sampling algorithms for auto-regressive language models, which is not widely studied in the literature. We use the quality-diversity…
StereoSet: Measuring stereotypical bias in pretrained language models
Moin Nadeem, Anna Bethke, Siva Reddy
A stereotype is an over-generalized belief about a particular group of people, e.g., Asians are good at math or Asians are bad drivers. Such beliefs (biases) are known to hurt targ…
FAKTA: An Automatic End-to-End Fact Checking System
Moin Nadeem, Wei Fang, Brian Xu +2
We present FAKTA which is a unified framework that integrates various components of a fact checking process: document retrieval from media sources with various types of reliability…
Context-Aware Systems for Sequential Item Recommendation
Moin Nadeem, Dustin Stansbury, Shane Mooney
Quizlet is the most popular online learning tool in the United States, and is used by over 2/3 of high school students, and 1/2 of college students. With more than 95% of Quizlet u…
Automating Network Error Detection using Long-Short Term Memory Networks
Moin Nadeem, Vibhor Nigam, Dimosthenis Anagnostopoulos +1
In this work, we investigate the current flaws with identifying network-related errors, and examine how K-Means and Long-Short Term Memory Networks solve these problems. We demonst…