65 citations · 292 across the 21 of their papers we have counts for
21 papers
Toloka Visual Question Answering Benchmark
Dmitry Ustalov, Nikita Pavlichenko, Sergey Koshelev +2
In this paper, we present Toloka Visual Question Answering, a new crowdsourced dataset allowing comparing performance of machine learning systems against human level of expertise i…
Best Prompts for Text-to-Image Models and How to Find Them
Nikita Pavlichenko, Dmitry Ustalov
Recent progress in generative models, especially in text-guided diffusion models, has enabled the production of aesthetically-pleasing imagery resembling the works of professional…
Clustering Without Knowing How To: Application and Evaluation
Daniil Likhobaba, Daniil Fedulov, Dmitry Ustalov
Crowdsourcing allows running simple human intelligence tasks on a large crowd of workers, enabling solving problems for which it is difficult to formulate an algorithm or train a m…
IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons
Nikita Pavlichenko, Dmitry Ustalov
Today, comprehensive evaluation of large-scale machine learning models is possible thanks to the open datasets produced using crowdsourcing, such as SQuAD, MS COCO, ImageNet, Super…
Learning from Crowds with Crowd-Kit
Dmitry Ustalov, Nikita Pavlichenko, Boris Tseitlin
This paper presents Crowd-Kit, a general-purpose computational quality control toolkit for crowdsourcing. Crowd-Kit provides efficient and convenient implementations of popular qua…
CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription
Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov
Domain-specific data is the crux of the successful transfer of machine learning systems from benchmarks to real life. In simple problems such as image classification, crowdsourcing…
Word Sense Disambiguation for 158 Languages using Word Embeddings Only
Varvara Logacheva, Denis Teslenko, Artem Shelmanov +7
Disambiguation of word senses in context is easy for humans, but is a major challenge for automatic approaches. Sophisticated supervised and knowledge-based models were developed t…
HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings
Saba Anwar, Dmitry Ustalov, Nikolay Arefyev +3
We present our system for semantic frame induction that showed the best performance in Subtask B.1 and finished as the runner-up in Subtask A of the SemEval 2019 Task 2 on unsuperv…
Unsupervised Sense-Aware Hypernymy Extraction
Dmitry Ustalov, Alexander Panchenko, Chris Biemann +1
In this paper, we show how unsupervised sense representations can be used to improve hypernymy extraction. We present a method for extracting disambiguated hypernymy relationships…
Watset: Local-Global Graph Clustering with Applications in Sense and Frame Induction
Dmitry Ustalov, Alexander Panchenko, Chris Biemann +1
We present a detailed theoretical and computational analysis of the Watset meta-algorithm for fuzzy graph clustering, which has been found to be widely applicable in a variety of d…
Unsupervised Semantic Frame Induction using Triclustering
Dmitry Ustalov, Alexander Panchenko, Andrei Kutuzov +2
We use dependency triples automatically extracted from a Web-scale corpus to perform unsupervised semantic frame induction. We cast the frame induction problem as a triclustering p…
An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages
Dmitry Ustalov, Denis Teslenko, Alexander Panchenko +3
In this paper, we present Watasense, an unsupervised system for word sense disambiguation. Given a sentence, the system chooses the most relevant sense of each input word with resp…
RUSSE: The First Workshop on Russian Semantic Similarity
Alexander Panchenko, Natalia Loukachevitch, Dmitry Ustalov +3
The paper gives an overview of the Russian Semantic Similarity Evaluation (RUSSE) shared task held in conjunction with the Dialogue 2015 conference. There exist a lot of comparativ…
RUSSE'2018: A Shared Task on Word Sense Induction for the Russian Language
Alexander Panchenko, Anastasiya Lopukhina, Dmitry Ustalov +4
The paper describes the results of the first shared task on word sense induction (WSI) for the Russian language. While similar shared tasks were conducted in the past for some Roma…
Improving Hypernymy Extraction with Distributional Semantic Classes
Alexander Panchenko, Dmitry Ustalov, Stefano Faralli +2
In this paper, we show how distributionally-induced semantic classes can be helpful for extracting hypernyms. We present methods for inducing sense-aware semantic classes using dis…
Human and Machine Judgements for Russian Semantic Relatedness
Alexander Panchenko, Dmitry Ustalov, Nikolay Arefyev +4
Semantic relatedness of terms represents similarity of meaning by a numerical score. On the one hand, humans easily make judgments about semantic relatedness. On the other hand, th…
Fighting with the Sparsity of Synonymy Dictionaries
Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann +1
Graph-based synset induction methods, such as MaxMax and Watset, induce synsets by performing a global clustering of a synonymy graph. However, such methods are sensitive to the st…
Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation
Alexander Panchenko, Fide Marten, Eugen Ruppert +4
Interpretability of a predictive model is a powerful feature that gains the trust of users in the correctness of the predictions. In word sense disambiguation (WSD), knowledge-base…
Negative Sampling Improves Hypernymy Extraction Based on Projection Learning
Dmitry Ustalov, Nikolay Arefyev, Chris Biemann +1
We present a new approach to extraction of hypernyms based on projection learning and word embeddings. In contrast to classification-based approaches, projection-based methods requ…
Watset: Automatic Induction of Synsets from a Graph of Synonyms
Dmitry Ustalov, Alexander Panchenko, Chris Biemann
This paper presents a new graph-based approach that induces synsets using synonymy dictionaries and word embeddings. First, we build a weighted graph of synonyms extracted from com…
Towards crowdsourcing and cooperation in linguistic resources
Dmitry Ustalov
Linguistic resources can be populated with data through the use of such approaches as crowdsourcing and gamification when motivated people are involved. However, current crowdsourc…