Numerical Optimizations for Weighted Low-rank Estimation on Language Model

Ting Hua, Yen-Chang Hsu, Felicity Wang +3

Singular value decomposition (SVD) is one of the most popular compression methods that approximate a target matrix with smaller matrices. However, standard SVD treats the parameter…