34 citations · 57 across the 5 of their papers we have counts for
5 papers
Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent
Wei Zhang, Mingrui Liu, Yu Feng +3
Distributed Deep Learning (DDL) is essential for large-scale Deep Learning (DL) training. Synchronous Stochastic Gradient Descent (SSGD) 1 is the de facto DDL optimization method.…
Asynchronous Decentralized Distributed Training of Acoustic Models
Xiaodong Cui, Wei Zhang, Abdullah Kayi +5
Large-scale distributed training of deep acoustic models plays an important role in today's high-performance automatic speech recognition (ASR). In this paper we investigate a vari…
Improving Efficiency in Large-Scale Decentralized Distributed Training
Wei Zhang, Xiaodong Cui, Abdullah Kayi +9
Decentralized Parallel SGD (D-PSGD) and its asynchronous variant Asynchronous Parallel SGD (AD-PSGD) is a family of distributed learning algorithms that have been demonstrated to p…
Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets
Mingrui Liu, Youssef Mroueh, Jerret Ross +4
Adaptive gradient algorithms perform gradient-based updates using the history of gradients and are ubiquitous in training deep neural networks. While adaptive gradient methods theo…
A Decentralized Parallel Algorithm for Training Generative Adversarial Nets
Mingrui Liu, Wei Zhang, Youssef Mroueh +4
Generative Adversarial Networks (GANs) are a powerful class of generative models in the deep learning community. Current practice on large-scale GAN training utilizes large models…