4 papers
A Grounded and Decomposed Framework for Relation-Level Hallucination Evaluation in Abstractive Summarization
Praveen Kumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala +1
Abstractive text summarization systems frequently generate fluent yet unfaithful summaries by fabricating or distorting relationships between entities and events. Such relation-lev…
Abstractiveness Metrics for Evaluating Text Summarization: A Refined Formulation with Empirical Validation
Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala
Quantifying abstractiveness in generated summaries is essential for evaluating summarization models beyond surface-level metrics like ROUGE. We introduce Reference Abstraction (RA)…
Bridging the Data Gap: Creating a Hindi Text Summarization Dataset from the English XSUM
Praveenkumar Katwe, RakeshChandra Balabantaray, Kaliprasad Vittala
Current advancements in Natural Language Processing (NLP) have largely favored resource-rich languages, leaving a significant gap in high-quality datasets for low-resource language…
Reducing Hallucinations in Summarization via Reinforcement Learning with Entity Hallucination Index
Praveenkumar Katwe, Rakesh Chandra, Balabantaray Kali +1
Reducing hallucinations in abstractive summarization remains a critical challenge for deploying language models (LMs) in real-world settings. In this work, we introduce a rewarddri…