6 citations · 14 across the 15 of their papers we have counts for
5 papers · 1 filter
Namesakes: Ambiguously Named Entities from Wikipedia and News
Oleg Vasilyev, Aysu Altun, Nidhi Vyas +3
We present Namesakes, a dataset of ambiguously named entities obtained from English-language Wikipedia and news articles. It consists of 58862 mentions of 4148 unique entities and…
Does Summary Evaluation Survive Translation to Other Languages?
Spencer Braun, Oleg Vasilyev, Neslihan Iskender +1
The creation of a quality summarization dataset is an expensive, time-consuming effort, requiring the production and evaluation of summaries by both trained humans and machines. If…
Towards Human-Free Automatic Quality Evaluation of German Summarization
Neslihan Iskender, Oleg Vasilyev, Tim Polzehl +2
Evaluating large summarization corpora using humans has proven to be expensive from both the organizational and the financial perspective. Therefore, many automatic evaluation metr…
Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings
Oleg Vasilyev, John Bohannon
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is designed to find and count all possible minute inconsistencies…
Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluation
Nicholas Egan, Oleg Vasilyev, John Bohannon
The goal of a summary is to concisely state the most important information in a document. With this principle in mind, we introduce new reference-free summary evaluation metrics th…