873 citations · 5.4k across the 120 of their papers we have counts for
16 papers · 1 filter
CADA: Communication-Adaptive Distributed Adam
Tianyi Chen, Ziye Guo, Yuejiao Sun +1
Stochastic gradient descent (SGD) has taken the stage as the primary workhorse for large-scale machine learning. It is often used with its adaptive variants such as AdaGrad, Adam,…
Hybrid Federated Learning: Algorithms and Implementation
Xinwei Zhang, Wotao Yin, Mingyi Hong +1
Federated learning (FL) is a recently proposed distributed machine learning paradigm dealing with distributed and private data sets. Based on the data partition pattern, FL is ofte…
Attentional-Biased Stochastic Gradient Descent
Qi Qi, Yi Xu, Rong Jin +2
In this paper, we present a simple yet effective provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple mod…
A One-bit, Comparison-Based Gradient Estimator
HanQin Cai, Daniel Mckenzie, Wotao Yin +1
We study zeroth-order optimization for convex functions where we further assume that function evaluations are unavailable. Instead, one only has access to a $\textit{comparison ora…
An Improved Analysis of Stochastic Gradient Descent with Momentum
Yanli Liu, Yuan Gao, Wotao Yin
SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise manner. De…
Decentralized Learning with Lazy and Approximate Dual Gradients
Yanli Liu, Yuejiao Sun, Wotao Yin
This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neig…