5 papers
Don't Scroll Back: Missing-Evidence Memory for Streaming Dialogue Summarization
Hyangsuk Min, Hwanjun Song
Users of modern platforms repeatedly need summaries of recent dialogue, but the window rarely contains enough context to be interpreted on its own. We formalize this setting as str…
Completing Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarks
Minjeong Ban, Jeonghwan Choi, Hyangsuk Min +4
Information retrieval (IR) evaluation remains challenging due to incomplete IR benchmark datasets that contain unlabeled relevant chunks. While LLMs and LLM-human hybrid strategies…
Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts
Jiaqi Deng, Yuho Lee, Nicole Hee-Yeon Kim +5
We introduce HAMLET, a holistic and automated framework for evaluating the long-context comprehension of large language models (LLMs). HAMLET structures source texts into a three-l…
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
Hyangsuk Min, Yuho Lee, Minjeong Ban +6
Evaluation frameworks for text summarization have evolved in terms of both domain coverage and metrics. However, existing benchmarks still lack domain-specific assessment criteria,…
ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedback
Taewon Yun, Jihwan Oh, Hyangsuk Min +4
Summarization refinement faces challenges when extending to multi-dimension. In this paper, we introduce ReFeed, a powerful summarization refinement pipeline that enhances multiple…