3 citations · 3 across the 2 of their papers we have counts for
4 papers · 1 filter
Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation
Sanjana Ramprasad, Byron C. Wallace
Modern LLMs can now produce highly readable abstractive summaries, to the point that traditional automated metrics for evaluating summary quality, such as ROUGE, have saturated. Ho…
Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends
Sanjana Ramprasad, Elisa Ferracane, Zachary C. Lipton
Recent advancements in large language models (LLMs) have considerably advanced the capabilities of summarization systems. However, they continue to face concerns about hallucinatio…
Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains
Sanjana Ramprasad, Kundan Krishna, Zachary C Lipton +1
Recent work has shown that large language models (LLMs) are capable of generating summaries zero-shot (i.e., without explicit supervision) that, under human assessment, are often c…
GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
Kundan Krishna, Sanjana Ramprasad, Prakhar Gupta +3
LLMs can generate factually incorrect statements even when provided access to reference documents. Such errors can be dangerous in high-stakes applications (e.g., document-grounded…