Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality

Zhenglun Kong, Yize Li, Fanhu Zeng +7

In Transformer architectures, tokens\textemdash discrete units derived from raw data\textemdash are formed by segmenting inputs into fixed-length chunks. Each token is then mapped…