Showing 2025Show all
2 papers · 1 filter
cs.CL2025
Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic
Yuval Reif, Guy Kaplan, Roy Schwartz
Large language models (LLMs) often encode word-form variation (e.g., walk vs. walked) as linear directions in the embedding space. However, standard tokenization algorithms treat s…
cs.CL2025
SpeLLM: Character-Level Multi-Head Decoding
Amit Ben-Artzy, Roy Schwartz
Scaling LLM vocabulary is often used to reduce input sequence length and alleviate attention's quadratic cost. Yet, current LLM architectures impose a critical bottleneck to this p…