125 citations · 182 across the 14 of their papers we have counts for
9 papers · 1 filter
Differentially Private Estimation of Hawkes Process
Simiao Zuo, Tianyi Liu, Tuo Zhao +1
Point process models are of great importance in real world applications. In certain critical applications, estimation of point process models involves large amounts of sensitive pe…
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably
Tianyi Liu, Yan Li, Enlu Zhou +1
We investigate the role of noise in optimization algorithms for learning over-parameterized models. Specifically, we consider the recovery of a rank one matrix $Y^*\in R^{d\times d…
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Guojing Cong, Tianyi Liu
Momentum method has been used extensively in optimizers for deep learning. Recent studies show that distributed training through K-step averaging has many nice properties. We propo…
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization
Tianyi Liu, Yan Li, Song Wei +2
Numerous empirical evidences have corroborated the importance of noise in nonconvex optimization problems. The theory behind such empirical observations, however, is still largely…
On Computation and Generalization of Generative Adversarial Imitation Learning
Minshuo Chen, Yizhou Wang, Tianyi Liu +4
Generative Adversarial Imitation Learning (GAIL) is a powerful and practical approach for learning sequential decision-making policies. Different from Reinforcement Learning (RL),…
Towards Understanding the Importance of Shortcut Connections in Residual Networks
Tianyi Liu, Minshuo Chen, Mo Zhou +3
Residual Network (ResNet) is undoubtedly a milestone in deep learning. ResNet is equipped with shortcut connections between layers, and exhibits efficient training using simple fir…
Towards Understanding the Importance of Noise in Training Neural Networks
Mo Zhou, Tianyi Liu, Yan Li +3
Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largel…
Towards Understanding Acceleration Tradeoff between Momentum and Asynchrony in Nonconvex Stochastic Optimization
Tianyi Liu, Shiyang Li, Jianping Shi +2
Asynchronous momentum stochastic gradient descent algorithms (Async-MSGD) is one of the most popular algorithms in distributed machine learning. However, its convergence properties…
A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization
Tianyi Liu, Zhehui Chen, Enlu Zhou +1
Momentum Stochastic Gradient Descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning, e.g., training deep neural networks, vari…