4 citations · 7 across the 5 of their papers we have counts for
5 papers
Bias in News Summarization: Measures, Pitfalls and Corpora
Julius Steen, Katja Markert
Summarization is an important application of large language models (LLMs). Most previous evaluation of summarization models has focused on their content selection, faithfulness, gr…
AMR4NLI: Interpretable and robust NLI measures from semantic graphs
Juri Opitz, Shira Wein, Julius Steen +2
The task of natural language inference (NLI) asks whether a given premise (expressed in NL) entails a given NL hypothesis. NLI benchmarks contain human ratings of entailment, but t…
With a Little Push, NLI Models can Robustly and Efficiently Predict Faithfulness
Julius Steen, Juri Opitz, Anette Frank +1
Conditional language models still generate unfaithful output that is not supported by their input. These unfaithful generations jeopardize trust in real-world applications such as…
How to Find Strong Summary Coherence Measures? A Toolbox and a Comparative Study for Summary Coherence Measure Evaluation
Julius Steen, Katja Markert
Automatically evaluating the coherence of summaries is of great significance both to enable cost-efficient summarizer evaluation and as a tool for improving coherence by selecting…
How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation
Julius Steen, Katja Markert
Manual evaluation is essential to judge progress on automatic text summarization. However, we conduct a survey on recent summarization system papers that reveals little agreement o…