182 citations · 191 across the 11 of their papers we have counts for
14 papers
PLURAL: A Global Dataset for Value Alignment
Dhruv Agarwal, Anya Shukla, Tanya Goyal +1
Large language models (LLMs) are used worldwide, yet disproportionately reflect Western values, limiting their ability to represent diverse value systems. We introduce PLURAL, a la…
D2PO: Discriminator-Guided DPO with Response Evaluation Models
Prasann Singhal, Nathan Lambert, Scott Niekum +2
Varied approaches for aligning language models have been proposed, including supervised fine-tuning, RLHF, and direct optimization methods such as DPO. Although DPO has rapidly gai…
A Long Way to Go: Investigating Length Correlations in RLHF
Prasann Singhal, Tanya Goyal, Jiacheng Xu +1
Great success has been reported using Reinforcement Learning from Human Feedback (RLHF) to align large language models, with open preference datasets enabling wider experimentation…
WiCE: Real-World Entailment for Claims in Wikipedia
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez +1
Textual entailment models are increasingly applied in settings like fact-checking, presupposition verification in question answering, or summary evaluation. However, these represen…
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization
Ryo Kamoi, Tanya Goyal, Greg Durrett
Despite recent progress in abstractive summarization, models often generate summaries with factual errors. Numerous approaches to detect these errors have been proposed, the most p…
News Summarization and Evaluation in the Era of GPT-3
Tanya Goyal, Junyi Jessy Li, Greg Durrett
The recent success of prompting large language models like GPT-3 has led to a paradigm shift in NLP research. In this paper, we study its impact on text summarization, focusing on…
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
Liyan Tang, Tanya Goyal, Alexander R. Fabbri +6
The propensity of abstractive summarization models to make factual errors has been studied extensively, including design of metrics to detect factual errors and annotation of error…
SNaC: Coherence Error Detection for Narrative Summarization
Tanya Goyal, Junyi Jessy Li, Greg Durrett
Progress in summarizing long texts is inhibited by the lack of appropriate evaluation frameworks. When a long summary must be produced to appropriately cover the facets of that tex…
Training Dynamics for Text Summarization Models
Tanya Goyal, Jiacheng Xu, Junyi Jessy Li +1
Pre-trained language models (e.g. BART) have shown impressive results when fine-tuned on large summarization datasets. However, little is understood about this fine-tuning process,…
HydraSum: Disentangling Stylistic Features in Text Summarization using Multi-Decoder Models
Tanya Goyal, Nazneen Fatema Rajani, Wenhao Liu +1
Summarization systems make numerous "decisions" about summary properties during inference, e.g. degree of copying, specificity and length of outputs, etc. However, these are implic…
Annotating and Modeling Fine-grained Factuality in Summarization
Tanya Goyal, Greg Durrett
Recent pre-trained abstractive summarization systems have started to achieve credible performance, but a major barrier to their use in practice is their propensity to output summar…
Evaluating Factuality in Generation with Dependency-level Entailment
Tanya Goyal, Greg Durrett
Despite significant progress in text generation models, a serious limitation is their tendency to produce text that is factually inconsistent with information in the input. Recent…
Neural Syntactic Preordering for Controlled Paraphrase Generation
Tanya Goyal, Greg Durrett
Paraphrasing natural language sentences is a multifaceted process: it might involve replacing individual words or short phrases, local rearrangement of content, or high-level restr…
Embedding time expressions for deep temporal ordering models
Tanya Goyal, Greg Durrett
Data-driven models have demonstrated state-of-the-art performance in inferring the temporal ordering of events in text. However, these models often overlook explicit temporal signa…