3 papers
cs.CL2026
LangMAP: A Language-Adaptive Approach to Tokenization
Clara Meister, Suchir Salhan, Andrzej Szablewski +3
Language-specific tokenizers improve tokenization quality and the downstream performance of models on those languages. However, using such a tokenizer comes at a cost: either a new…
cs.CL2025
Causal Estimation of Tokenisation Bias
Pietro Lesci, Clara Meister, Thomas Hofmann +2
Modern language models are typically trained over subword sequences, but ultimately define probabilities over character-strings. Ideally, the choice of the tokeniser -- which maps…
cs.CL2025
Self-Training Large Language Models for Tool-Use Without Demonstrations
Ne Luo, Aryo Pradipta Gema, Xuanli He +3
Large language models (LLMs) remain prone to factual inaccuracies and computational errors, including hallucinations and mistakes in mathematical reasoning. Recent work augmented L…