873 citations · 5.4k across the 120 of their papers we have counts for
19 papers · 1 filter
A Novel Convergence Analysis for Algorithms of the Adam Family
Zhishuai Guo, Yi Xu, Wotao Yin +2
Since its invention in 2014, the Adam optimizer has received tremendous attention. On one hand, it has been widely used in deep learning and many variants have been proposed, while…
From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization
Qiuwei Li, Daniel McKenzie, Wotao Yin
The standard simplex in R^n, also known as the probability simplex, is the set of nonnegative vectors whose entries sum up to 1. They frequently appear as constraints in optimizati…
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning
Bicheng Ying, Kun Yuan, Hanbin Hu +2
Decentralized algorithm is a form of computation that achieves a global goal through local dynamics that relies on low-cost communication between directly-connected agents. On larg…
Hyperparameter Tuning is All You Need for LISTA
Xiaohan Chen, Jialin Liu, Zhangyang Wang +1
Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network. It has had great succes…
Exponential Graph is Provably Efficient for Decentralized Deep Training
Bicheng Ying, Kun Yuan, Yiming Chen +3
Decentralized SGD is an emerging training method for deep learning known for its much less (thus faster) communication per iteration, which relaxes the averaging step in parallel S…
Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection
HanQin Cai, Jialin Liu, Wotao Yin
Robust principal component analysis (RPCA) is a critical tool in modern machine learning, which detects outliers in the task of low-rank matrix reconstruction. In this paper, we pr…