Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory

Kohei Nishikawa, Koki Shimizu, Hiroki Hashiguchi

This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight mat…