71 citations · 73 across the 3 of their papers we have counts for
3 papers
Cuttlefish: Low-Rank Model Training without All the Tuning
Hongyi Wang, Saurabh Agarwal, Pongsakorn U-chupala +3
Recent research has shown that training low-rank neural networks can effectively reduce the total number of trainable parameters without sacrificing predictive accuracy, resulting…
MRL: Learning to Mix with Attention and Convolutions
Shlok Mohta, Hisahiro Suganuma, Yoshiki Tanaka
In this paper, we present a new neural architectural block for the vision domain, named Mixing Regionally and Locally (MRL), developed with the aim of effectively and efficiently m…
Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flash
Hiroaki Mikami, Hisahiro Suganuma, Pongsakorn U-chupala +2
Scaling the distributed deep learning to a massive GPU cluster level is challenging due to the instability of the large mini-batch training and the overhead of the gradient synchro…