5 citations · 9 across the 11 of their papers we have counts for
11 papers · 1 filter
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
Hyangsuk Min, Yuho Lee, Minjeong Ban +6
Evaluation frameworks for text summarization have evolved in terms of both domain coverage and metrics. However, existing benchmarks still lack domain-specific assessment criteria,…
Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation
Mahnaz Koupaee, Jake W. Vincent, Saab Mansour +9
Faithfulness evaluators based on large language models (LLMs) are often fooled by the fluency of the text and struggle with identifying errors in the summaries. We propose an appro…
Learning to Verify Summary Facts with Fine-Grained LLM Feedback
Jihwan Oh, Jeonghwan Choi, Nicole Hee-Yeon Kim +2
Training automatic summary fact verifiers often faces the challenge of a lack of human-labeled data. In this paper, we explore alternative way of leveraging Large Language Model (L…
Learning to Summarize from LLM-generated Feedback
Hwanjun Song, Taewon Yun, Yuho Lee +4
Developing effective text summarizers remains a challenge due to issues like hallucinations, key information omissions, and verbosity in LLM-generated summaries. This work explores…
UniSumEval: Towards Unified, Fine-Grained, Multi-Dimensional Summarization Evaluation for LLMs
Yuho Lee, Taewon Yun, Jason Cai +2
Existing benchmarks for summarization quality evaluation often lack diverse input scenarios, focus on narrowly defined dimensions (e.g., faithfulness), and struggle with subjective…
FineSurE: Fine-grained Summarization Evaluation using LLMs
Hwanjun Song, Hang Su, Igor Shalyminov +2
Automated evaluation is crucial for streamlining text summarization benchmarking and model development, given the costly and time-consuming nature of human evaluation. Traditional…