72 citations · 98 across the 13 of their papers we have counts for
4 papers · 1 filter
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
Yuki Takezawa, Han Bao, Kenta Niwa +2
SGD with momentum is one of the key components for improving the performance of neural networks. For decentralized learning, a straightforward approach using momentum is Distribute…
Communication Compression for Decentralized Learning with Operator Splitting Methods
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In decentralized learning, operator splitting methods using a primal-dual formulation (e.g., the Edge-Consensus Learning (ECL)) has been shown to be robust to heterogeneous data an…
Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning, but also for preserving privacy. One of the key challenges in dece…
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
We make contributions towards improving adaptive-optimizer performance. Our improvements are based on suppression of the range of adaptive stepsizes in the AdaBelief optimizer. Fir…