1.8k citations · 9.5k across the 166 of their papers we have counts for
4 papers · 1 filter
Stochastic Second-order Methods for Non-convex Optimization with Inexact Hessian and Gradient
Liu Liu, Xuanqing Liu, Cho-Jui Hsieh +1
Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points. Each iter…
Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem
Liu Liu, Ji Liu, Cho-Jui Hsieh +1
In this paper, we consider the convex and non-convex composition problem with the structure , where $G( x )=\frac{1}{n}\sum\…
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
Xiangru Lian, Ce Zhang, Huan Zhang +3
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are built in a centralized fashion. One bottleneck of centralized algorithms lies on high communi…
A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order
Xiangru Lian, Huan Zhang, Cho-Jui Hsieh +2
Asynchronous parallel optimization received substantial successes and extensive attention recently. One of core theoretical questions is how much speedup (or benefit) the asynchron…