3 papers
cs.CL2025
DVAGen: Dynamic Vocabulary Augmented Generation
Wei Du, Nuowei Liu, Jie Wang +4
Language models trained with a fixed vocabulary struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. Exist…
cs.CL2024
Generation with Dynamic Vocabulary
Yanting Liu, Tao Ji, Changzhi Sun +2
We introduce a new dynamic vocabulary for language models. It can involve arbitrary text spans during generation. These text spans act as basic generation bricks, akin to tokens in…
cs.CL2024
Length Generalization of Causal Transformers without Position Encoding
Jie Wang, Tao Ji, Yuanbin Wu +5
Generalizing to longer sentences is important for recent Transformer-based language models. Besides algorithms manipulating explicit position features, the success of Transformers…