14 citations · 31 across the 15 of their papers we have counts for
15 papers
Anderson Acceleration Without Restart: A Novel Method with -Step Super Quadratic Convergence Rate
Haishan Ye, Dachao Lin, Xiangyu Chang +1
In this paper, we propose a novel Anderson's acceleration method to solve nonlinear equations, which does \emph{not} require a restart strategy to achieve numerical stability. We p…
On the Convergence of Projected Policy Gradient for Any Constant Step Sizes
Jiacai Liu, Wenye Li, Dachao Lin +2
Projected policy gradient (PPG) is a basic policy optimization method in reinforcement learning. Given access to exact policy evaluations, previous studies have established the sub…
Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
Dachao Lin, Yuze Han, Haishan Ye +1
We study finite-sum distributed optimization problems involving a master node and local nodes under the popular -similarity and -strong convexity conditions. We propose…
On the Convergence of Policy in Unregularized Policy Mirror Descent
Dachao Lin, Zhihua Zhang
In this short note, we give the convergence analysis of the policy in the recent famous policy mirror descent (PMD). We mainly consider the unregularized setting following [11] wit…
Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer
Kun Chen, Dachao Lin, Zhihua Zhang
In this paper, we follow Eftekhari's work to give a non-local convergence analysis of deep linear networks. Specifically, we consider optimizing deep linear networks which have a l…
Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations
Haishan Ye, Dachao Lin, Zhihua Zhang
In this paper, we propose the greedy and random Broyden's method for solving nonlinear equations. Specifically, the greedy method greedily selects the direction to maximize a certa…
Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
Dachao Lin, Haishan Ye, Zhihua Zhang
In this paper, we study the explicit superlinear convergence rates of quasi-Newton methods. We particularly focus on the classical Broyden's method for solving nonlinear equations.…
Explicit Superlinear Convergence Rates of The SR1 Algorithm
Haishan Ye, Dachao Lin, Zhihua Zhang +1
We study the convergence rate of the famous Symmetric Rank-1 (SR1) algorithm which has wide applications in different scenarios. Although it has been extensively investigated, SR1…
Directional Convergence Analysis under Spherically Symmetric Distribution
Dachao Lin, Zhihua Zhang
We consider the fundamental problem of learning linear predictors (i.e., separable datasets with zero margin) using neural networks with gradient flow or gradient descent. Under th…
Meta-Regularization: An Approach to Adaptive Choice of the Learning Rate in Gradient Descent
Guangzeng Xie, Hao Jin, Dachao Lin +1
We propose \textit{Meta-Regularization}, a novel approach for the adaptive choice of the learning rate in first-order gradient descent methods. Our approach modifies the objective…
Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods
Dachao Lin, Haishan Ye, Zhihua Zhang
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In…
On the Landscape of One-hidden-layer Sparse Networks and Beyond
Dachao Lin, Ruoyu Sun, Zhihua Zhang
Sparse neural networks have received increasing interest due to their small size compared to dense networks. Nevertheless, most existing works on neural network theory have focused…
Optimal Quantization for Batch Normalization in Neural Network Deployments and Beyond
Dachao Lin, Peiqin Sun, Guangzeng Xie +2
Quantized Neural Networks (QNNs) use low bit-width fixed-point numbers for representing weight parameters and activations, and are often used in real-world applications due to thei…
Towards Understanding the Importance of Noise in Training Neural Networks
Mo Zhou, Tianyi Liu, Yan Li +3
Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largel…
Towards Better Generalization: BP-SVRG in Training Deep Neural Networks
Hao Jin, Dachao Lin, Zhihua Zhang
Stochastic variance-reduced gradient (SVRG) is a classical optimization method. Although it is theoretically proved to have better convergence performance than stochastic gradient…