3 citations · 6 across the 6 of their papers we have counts for
10 papers
BabyBear: Cheap inference triage for expensive language models
Leila Khalili, Yao You, John Bohannon
Transformer language models provide superior accuracy over previous models but they are computationally and environmentally expensive. Borrowing the concept of model cascading from…
Consistency and Coherence from Points of Contextual Similarity
Oleg Vasilyev, John Bohannon
Factual consistency is one of important summary evaluation dimensions, especially as summary generation becomes more fluent and coherent. The ESTIME measure, recently proposed spec…
Towards Human-Free Automatic Quality Evaluation of German Summarization
Neslihan Iskender, Oleg Vasilyev, Tim Polzehl +2
Evaluating large summarization corpora using humans has proven to be expensive from both the organizational and the financial perspective. Therefore, many automatic evaluation metr…
Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings
Oleg Vasilyev, John Bohannon
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is designed to find and count all possible minute inconsistencies…
Primer AI's Systems for Acronym Identification and Disambiguation
Nicholas Egan, John Bohannon
The prevalence of ambiguous acronyms make scientific documents harder to understand for humans and machines alike, presenting a need for models that can automatically identify acro…
Is human scoring the best criteria for summary evaluation?
Oleg Vasilyev, John Bohannon
Normally, summary quality measures are compared with quality scores produced by human annotators. A higher correlation with human scores is considered to be a fair indicator of a b…