Showing 2026Show all
3 papers · 1 filter
cs.SE2026
Compressed code: the hidden effects of quantization and distillation on programming tokens
Viacheslav Siniaev, Iaroslav Chelombitko, Aleksey Komissarov
Large Language Models (LLMs) have demonstrated exceptional code generation capabilities, yet their token-level mechanisms remain underexplored, particularly in compressed models. T…
cs.CL2026
Subword-Based Comparative Linguistics across 242 Languages Using Wikipedia Glottosets
Iaroslav Chelombitko, Mika Hämäläinen, Aleksey Komissarov
We present a large-scale comparative study of 242 Latin and Cyrillic-script languages using subword-based methodologies. By constructing 'glottosets' from Wikipedia lexicons, we in…
cs.CL2026
SampoNLP: A Self-Referential Toolkit for Morphological Analysis of Subword Tokenizers
Iaroslav Chelombitko, Ekaterina Chelombitko, Aleksey Komissarov
The quality of subword tokenization is critical for Large Language Models, yet evaluating tokenizers for morphologically rich Uralic languages is hampered by the lack of clean morp…