Knowledge Distillation of Russian Language Models with Reduction of Vocabulary

Alina Kolesnikova, Yuri Kuratov, Vasily Konovalov +1

Today, transformer language models serve as a core component for majority of natural language processing tasks. Industrial application of such models requires minimization of compu…