Accelerating Transformers with Spectrum-Preserving Token Merging

Hoai-Chau Tran, Duy M. H. Nguyen, Duy M. Nguyen +7

Increasing the throughput of the Transformer architecture, a foundational component used in numerous state-of-the-art models for vision and language tasks (e.g., GPT, LLaVa), is an…