14 citations · 31 across the 15 of their papers we have counts for
6 papers · 1 filter
Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations
Haishan Ye, Dachao Lin, Zhihua Zhang
In this paper, we propose the greedy and random Broyden's method for solving nonlinear equations. Specifically, the greedy method greedily selects the direction to maximize a certa…
Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
Dachao Lin, Haishan Ye, Zhihua Zhang
In this paper, we study the explicit superlinear convergence rates of quasi-Newton methods. We particularly focus on the classical Broyden's method for solving nonlinear equations.…
Explicit Superlinear Convergence Rates of The SR1 Algorithm
Haishan Ye, Dachao Lin, Zhihua Zhang +1
We study the convergence rate of the famous Symmetric Rank-1 (SR1) algorithm which has wide applications in different scenarios. Although it has been extensively investigated, SR1…
Directional Convergence Analysis under Spherically Symmetric Distribution
Dachao Lin, Zhihua Zhang
We consider the fundamental problem of learning linear predictors (i.e., separable datasets with zero margin) using neural networks with gradient flow or gradient descent. Under th…
Meta-Regularization: An Approach to Adaptive Choice of the Learning Rate in Gradient Descent
Guangzeng Xie, Hao Jin, Dachao Lin +1
We propose \textit{Meta-Regularization}, a novel approach for the adaptive choice of the learning rate in first-order gradient descent methods. Our approach modifies the objective…
Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods
Dachao Lin, Haishan Ye, Zhihua Zhang
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In…