38 citations · 90 across the 18 of their papers we have counts for
4 papers · 1 filter
Summarization is (Almost) Dead
Xiao Pu, Mingqi Gao, Xiaojun Wan
How well can large language models (LLMs) generate summaries? We develop new datasets and conduct human evaluation experiments to evaluate the zero-shot generation capability of LL…
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework
Mingqi Gao, Xiaojun Wan, Jia Su +2
Factuality is important to dialogue summarization. Factual error correction (FEC) of model-generated summaries is one way to improve factuality. Current FEC evaluation that relies…
Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks
Xiao Pu, Mingqi Gao, Xiaojun Wan
Research on automated text summarization relies heavily on human and automatic evaluation. While recent work on human evaluation mainly adopted intrinsic evaluation methods, judgin…
Human-like Summarization Evaluation with ChatGPT
Mingqi Gao, Jie Ruan, Renliang Sun +3
Evaluating text summarization is a challenging problem, and existing evaluation metrics are far from satisfactory. In this study, we explored ChatGPT's ability to perform human-lik…