2 papers
cs.CL2025
T5Gemma 2: Seeing, Reading, and Understanding Longer
Biao Zhang, Paul Suganthan, Gaël Liu +17
We introduce T5Gemma 2, the next generation of the T5Gemma family of lightweight open encoder-decoder models, featuring strong multilingual, multimodal and long-context capabilitie…
cs.CL2024
Training LLMs over Neurally Compressed Text
Brian Lester, Jaehoon Lee, Alex Alemi +4
In this paper, we explore the idea of training large language models (LLMs) over highly compressed text. While standard subword tokenizers compress text by a small factor, neural t…