6 citations · 17 across the 16 of their papers we have counts for
18 papers
Preserving Multilingual Quality While Tuning Query Encoder on English Only
Oleg Vasilyev, Randy Sawaya, John Bohannon
A query encoder of a dual passage retrieval system can be tuned for specific types of queries or domains, while the precomputed and stored documents representations are kept intact…
How to Discern Important Urgent News?
Oleg Vasilyev, John Bohannon
We found that a simple property of clusters in a clustered dataset of news correlate strongly with importance and urgency of news (IUN) as assessed by LLM. We verified our finding…
Linear Cross-Lingual Mapping of Sentence Embeddings
Oleg Vasilyev, Fumika Isono, John Bohannon
Semantics of a sentence is defined with much less ambiguity than semantics of a single word, and we assume that it should be better preserved by translation to another language. If…
Neural Embeddings for Text
Oleg Vasilyev, John Bohannon
We propose a new kind of embedding for natural language text that deeply represents semantic meaning. Standard text embeddings use the outputs from hidden layers of a pretrained la…
BabyBear: Cheap inference triage for expensive language models
Leila Khalili, Yao You, John Bohannon
Transformer language models provide superior accuracy over previous models but they are computationally and environmentally expensive. Borrowing the concept of model cascading from…
Named Entity Linking with Entity Representation by Multiple Embeddings
Oleg Vasilyev, Alex Dauenhauer, Vedant Dharnidharka +1
We propose a simple and practical method for named entity linking (NEL), based on entity representation by multiple embeddings. To explore this method, and to review its dependency…
Consistency and Coherence from Points of Contextual Similarity
Oleg Vasilyev, John Bohannon
Factual consistency is one of important summary evaluation dimensions, especially as summary generation becomes more fluent and coherent. The ESTIME measure, recently proposed spec…
Namesakes: Ambiguously Named Entities from Wikipedia and News
Oleg Vasilyev, Aysu Altun, Nidhi Vyas +3
We present Namesakes, a dataset of ambiguously named entities obtained from English-language Wikipedia and news articles. It consists of 58862 mentions of 4148 unique entities and…
Does Summary Evaluation Survive Translation to Other Languages?
Spencer Braun, Oleg Vasilyev, Neslihan Iskender +1
The creation of a quality summarization dataset is an expensive, time-consuming effort, requiring the production and evaluation of summaries by both trained humans and machines. If…
Towards Human-Free Automatic Quality Evaluation of German Summarization
Neslihan Iskender, Oleg Vasilyev, Tim Polzehl +2
Evaluating large summarization corpora using humans has proven to be expensive from both the organizational and the financial perspective. Therefore, many automatic evaluation metr…
Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings
Oleg Vasilyev, John Bohannon
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is designed to find and count all possible minute inconsistencies…
Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluation
Nicholas Egan, Oleg Vasilyev, John Bohannon
The goal of a summary is to concisely state the most important information in a document. With this principle in mind, we introduce new reference-free summary evaluation metrics th…
Primer AI's Systems for Acronym Identification and Disambiguation
Nicholas Egan, John Bohannon
The prevalence of ambiguous acronyms make scientific documents harder to understand for humans and machines alike, presenting a need for models that can automatically identify acro…
Is human scoring the best criteria for summary evaluation?
Oleg Vasilyev, John Bohannon
Normally, summary quality measures are compared with quality scores produced by human annotators. A higher correlation with human scores is considered to be a fair indicator of a b…
Sensitivity of BLANC to human-scored qualities of text summaries
Oleg Vasilyev, Vedant Dharnidharka, Nicholas Egan +2
We explore the sensitivity of a document summary quality estimator, BLANC, to human assessment of qualities for the same summaries. In our human evaluations, we distinguish five su…
Zero-shot topic generation
Oleg Vasilyev, Kathryn Evans, Anna Venancio-Marques +1
We present an approach to generating topics using a model trained only for document title generation, with zero examples of topics given during training. We leverage features that…
Fill in the BLANC: Human-free quality estimation of document summaries
Oleg Vasilyev, Vedant Dharnidharka, John Bohannon
We present BLANC, a new approach to the automatic estimation of document summary quality. Our goal is to measure the functional performance of a summary with an objective, reproduc…
Headline Generation: Learning from Decomposable Document Titles
Oleg Vasilyev, Tom Grek, John Bohannon
We propose a novel method for generating titles for unstructured text documents. We reframe the problem as a sequential question-answering task. A deep neural network is trained on…