Cuttlefish: Low-Rank Model Training without All the Tuning

Hongyi Wang, Saurabh Agarwal, Pongsakorn U-chupala +3

Recent research has shown that training low-rank neural networks can effectively reduce the total number of trainable parameters without sacrificing predictive accuracy, resulting…