works on
embedding initialization 1model adaptation 1multilingual tokenization 1pretrained language models 1tokenizer expansion 1

From the 1 of 7 linked papers with an AI index.

activity
20242026
most citedLFM2 Technical Report

1 citations · 1 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2026

In-Place Tokenizer Expansion for Pre-trained LLMs

Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera +7

The paper proposes an in‑place tokenizer expansion method that continues a pre‑trained model’s BPE merges on multilingual data, reuses existing token embeddings, and initializes ne…