Memory-Efficient Vision Transformers: An Activation-Aware Mixed-Rank Compression Strategy

Seyedarmin Azizi, Mahdi Nazemi, Massoud Pedram

As Vision Transformers (ViTs) increasingly set new benchmarks in computer vision, their practical deployment on inference engines is often hindered by their significant memory band…