49 citations · 201 across the 21 of their papers we have counts for
8 papers · 1 filter
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
RuCoLA: Russian Corpus of Linguistic Acceptability
Vladislav Mikhailov, Tatiana Shamardina, Max Ryabinin +3
Linguistic acceptability (LA) attracts the attention of the research community due to its many uses, such as testing the grammatical knowledge of language models and filtering impl…
TAPE: Assessing Few-shot Russian Language Understanding
Ekaterina Taktasheva, Tatiana Shavrina, Alena Fenogenova +11
Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes. However, this fast-growing area lacks standardized evaluation…
Vote'n'Rank: Revision of Benchmarking with Social Choice Theory
Mark Rofin, Vladislav Mikhailov, Mikhail Florinskiy +5
The development of state-of-the-art systems in different applied areas of machine learning (ML) is driven by benchmarks, which have shaped the paradigm of evaluating generalisation…
Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
Tatiana Shamardina, Vladislav Mikhailov, Daniil Chernianskii +7
We present the shared task on artificial text detection in Russian, which is organized as a part of the Dialogue Evaluation initiative, held in 2022. The shared task dataset includ…
Acceptability Judgements via Examining the Topology of Attention Maps
Daniil Cherniavskii, Eduard Tulchinskii, Vladislav Mikhailov +7
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP. However, the ability of the attention heads to judge the grammatical acce…
mGPT: Few-Shot Learners Go Multilingual
Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova +3
Recent studies report that autoregressive language models can successfully solve many NLP tasks via zero- and few-shot learning paradigms, which opens up new possibilities for usin…
Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models
Alena Fenogenova, Maria Tikhonova, Vladislav Mikhailov +6
In the last year, new neural architectures and multilingual pre-trained models have been released for Russian, which led to performance evaluation problems across a range of langua…