42 citations · 163 across the 36 of their papers we have counts for
16 papers · 1 filter
High-Probability Last-Iterate Guarantees for Two-Point Gaussian Zeroth-Order Stochastic Gradient Descent
Haishan Ye
We establish a direct high-probability last-iterate guarantee for the standard same-sample two-point Gaussian zeroth-order SGD method in smooth, strongly convex stochastic optimiza…
High-Probability Guarantees for Random Zeroth-Order Gradient Descent on Smooth Functions
Haishan Ye
Randomized zeroth-order methods are classically analyzed in expectation, but a black-box Markov conversion can give misleading high-probability guarantees, in particular by forcing…
Stochastic Non-Smooth Non-Convex Optimization with Decision-Dependent Distributions
Chengchang Liu, Zongqi Wan, Haishan Ye +1
We study stochastic zeroth-order optimization with decision-dependent distributions, where the sampling law depends on the current decision and only noisy function values are avail…
High-Probability Guarantees for Random Zeroth-Order (Stochastic) Gradient Descent
Haishan Ye
Zeroth-order optimization aims to minimize an objective function using only function evaluations, and is therefore fundamental in black-box optimization, hyperparameter tuning, ban…
Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
Qihao Zhou, Haishan Ye, Luo Luo
This paper considers the distributed convex-concave minimax optimization under the second-order similarity. We propose stochastic variance-reduced optimistic gradient sliding (SVOG…
Optimal Decentralized Composite Optimization for Convex Functions
Haishan Ye, Xiangyu Chang
In this paper, we focus on the decentralized composite optimization for convex functions. Because of advantages such as robust to the network and no communication bottle-neck in th…
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold
Jun Chen, Haishan Ye, Mengmeng Wang +4
The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lowe…
Snap-Shot Decentralized Stochastic Gradient Tracking Methods
Haishan Ye, Xiangyu Chang
In decentralized optimization, agents form a network and only communicate with their neighbors, which gives advantages in data ownership, privacy, and scalability. At the same…
On the Complexity of Decentralized Smooth Nonconvex Finite-Sum Optimization
Luo Luo, Yunyan Bai, Lesi Chen +2
We study the decentralized optimization problem , where the local function on the -th a…
Decentralized Stochastic Variance Reduced Extragradient Method
Luo Luo, Haishan Ye
This paper studies decentralized convex-concave minimax optimization problems of the form , where is the number…
Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
Dachao Lin, Haishan Ye, Zhihua Zhang
In this paper, we study the explicit superlinear convergence rates of quasi-Newton methods. We particularly focus on the classical Broyden's method for solving nonlinear equations.…
Explicit Superlinear Convergence Rates of The SR1 Algorithm
Haishan Ye, Dachao Lin, Zhihua Zhang +1
We study the convergence rate of the famous Symmetric Rank-1 (SR1) algorithm which has wide applications in different scenarios. Although it has been extensively investigated, SR1…
Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods
Dachao Lin, Haishan Ye, Zhihua Zhang
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In…
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
Haishan Ye, Wei Xiong, Tong Zhang
This paper considers the decentralized composite optimization problem. We propose a novel decentralized variance-reduction proximal-gradient algorithmic framework, called PMGT-VR,…
Mirror Natural Evolution Strategies
Haishan Ye, Tong Zhang
Evolution Strategies such as CMA-ES (covariance matrix adaptation evolution strategy) and NES (natural evolution strategy) have been widely used in machine learning applications, w…
Revisiting Sub-sampled Newton Methods
Haishan Ye, Luo Luo, Zhihua Zhang
Many machine learning models depend on solving a large scale optimization problem. Recently, sub-sampled Newton methods have emerged to attract much attention for optimization due…