activity
20182024
most citedThe GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

52 citations · 158 across the 19 of their papers we have counts for

collaborators
Showing 2022Show all

7 papers · 1 filter

cs.CL2022★ 5 cited

mFACE: Multilingual Summarization with Factual Consistency Evaluation

Roee Aharoni, Shashi Narayan, Joshua Maynez +3

Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising result…

cs.CL2022★ 1 cited

Little Red Riding Hood Goes Around the Globe:Crosslingual Story Planning and Generation with Large Language Models

Evgeniia Razumovskaia, Joshua Maynez, Annie Louis +2

Previous work has demonstrated the effectiveness of planning for story generation exclusively in a monolingual setting focusing primarily on English. We consider whether planning b…

cs.CL2022★ 5 cited

Query Refinement Prompts for Closed-Book Long-Form Question Answering

Reinald Kim Amplayo, Kellie Webster, Michael Collins +2

Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings. While the former can be…

cs.CL2022★ 38 cited

Calibrating Sequence likelihood Improves Conditional Language Generation

Yao Zhao, Misha Khalman, Rishabh Joshi +3

Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences. While MLE trained mod…

cs.CL2022★ 5 cited

SMART: Sentences as Basic Units for Text Evaluation

Reinald Kim Amplayo, Peter J. Liu, Yao Zhao +1

Widely used evaluation metrics for text generation either do not work well with longer texts or fail to evaluate all aspects of text quality. In this paper, we introduce a new metr…

cs.CL2022★ 2 cited

Conditional Generation with a Question-Answering Blueprint

Shashi Narayan, Joshua Maynez, Reinald Kim Amplayo +6

The ability to convey relevant and faithful information is critical for many tasks in conditional generation and yet remains elusive for neural seq-to-seq models whose outputs ofte…