Mind the Approximation: Fisher-Weighted SVD Compression for ViTs

Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +7

Model compression is key to mitigate deployment challenges of ever growing machine learning models. In this area of research, singular value decomposition (SVD)-based compression o…