activity
20172026
most citedThe GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

52 citations · 62 across the 22 of their papers we have counts for

collaborators
Showing 2021Show all

5 papers · 1 filter

cs.CL2021

Truth-Conditional Captioning of Time Series Data

Harsh Jhamtani, Taylor Berg-Kirkpatrick

In this paper, we explore the task of automatically generating natural language descriptions of salient patterns in a time series, such as stock prices of a company over a week. A…

cs.CL2021

Investigating Robustness of Dialog Models to Popular Figurative Language Constructs

Harsh Jhamtani, Varun Gangal, Eduard Hovy +1

Humans often employ figurative language use in communication, including during interactions with dialog systems. Thus, it is important for real-world dialog systems to be able to h…

cs.CL2021

Unsupervised Enrichment of Persona-grounded Dialog with Background Stories

Bodhisattwa Prasad Majumder, Taylor Berg-Kirkpatrick, Julian McAuley +1

Humans often refer to personal narratives, life experiences, and events to make a conversation more engaging and rich. While persona-grounded dialog models are able to generate res…

cs.CL2021

Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation

Varun Gangal, Harsh Jhamtani, Eduard Hovy +1

Multiple different responses are often plausible for a given open domain dialog context. Prior work has shown the importance of having multiple valid reference responses for meanin…

cs.CL2021★ 52 cited

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal +53

We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of auto…