SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

Ernests Lavrinovics, Marco Letizia, Roy Janco +3

We present SigmaScale, a method for learning auxiliary scaling matrices $S$ to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression. Instea…