3 citations · 3 across the 4 of their papers we have counts for
8 papers
Consistency and Coherence from Points of Contextual Similarity
Oleg Vasilyev, John Bohannon
Factual consistency is one of important summary evaluation dimensions, especially as summary generation becomes more fluent and coherent. The ESTIME measure, recently proposed spec…
Towards Human-Free Automatic Quality Evaluation of German Summarization
Neslihan Iskender, Oleg Vasilyev, Tim Polzehl +2
Evaluating large summarization corpora using humans has proven to be expensive from both the organizational and the financial perspective. Therefore, many automatic evaluation metr…
Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings
Oleg Vasilyev, John Bohannon
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is designed to find and count all possible minute inconsistencies…
Is human scoring the best criteria for summary evaluation?
Oleg Vasilyev, John Bohannon
Normally, summary quality measures are compared with quality scores produced by human annotators. A higher correlation with human scores is considered to be a fair indicator of a b…
Sensitivity of BLANC to human-scored qualities of text summaries
Oleg Vasilyev, Vedant Dharnidharka, Nicholas Egan +2
We explore the sensitivity of a document summary quality estimator, BLANC, to human assessment of qualities for the same summaries. In our human evaluations, we distinguish five su…
Zero-shot topic generation
Oleg Vasilyev, Kathryn Evans, Anna Venancio-Marques +1
We present an approach to generating topics using a model trained only for document title generation, with zero examples of topics given during training. We leverage features that…