LayerCollapse: Adaptive compression of neural networks

Soheil Zibakhsh Shabgahi, Mohammad Sohail Shariff, Farinaz Koushanfar

Handling the ever-increasing scale of contemporary deep learning and transformer-based models poses a significant challenge. Overparameterized Transformer networks outperform prior…