2 papers
cs.CL2026
A Family of LLMs Liberated from Static Vocabularies
Aleph Alpha, :, Adnen Abdessaied +35
Tokenization is a central component of natural language processing in current large language models (LLMs), enabling models to convert raw text into processable units. Although lea…
cs.CL2023
Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantization
Björn Deiseroth, Max Meuer, Nikolas Gritsch +4
Large Language Models (LLMs) have reshaped natural language processing with their impressive capabilities. However, their ever-increasing size has raised concerns about their effec…