42 citations · 163 across the 36 of their papers we have counts for
36 papers
High-Probability Last-Iterate Guarantees for Two-Point Gaussian Zeroth-Order Stochastic Gradient Descent
Haishan Ye
We establish a direct high-probability last-iterate guarantee for the standard same-sample two-point Gaussian zeroth-order SGD method in smooth, strongly convex stochastic optimiza…
High-Probability Guarantees for Random Zeroth-Order Gradient Descent on Smooth Functions
Haishan Ye
Randomized zeroth-order methods are classically analyzed in expectation, but a black-box Markov conversion can give misleading high-probability guarantees, in particular by forcing…
Stochastic Non-Smooth Non-Convex Optimization with Decision-Dependent Distributions
Chengchang Liu, Zongqi Wan, Haishan Ye +1
We study stochastic zeroth-order optimization with decision-dependent distributions, where the sampling law depends on the current decision and only noisy function values are avail…
High-Probability Guarantees for Random Zeroth-Order (Stochastic) Gradient Descent
Haishan Ye
Zeroth-order optimization aims to minimize an objective function using only function evaluations, and is therefore fundamental in black-box optimization, hyperparameter tuning, ban…
Logarithmic High-Probability Regret for Online Convex Optimization with Two-Point Bandit Feedback
Haishan Ye
We study online convex optimization (OCO) with two-point bandit feedback against a non-anticipating adaptive adversary. In this setting, a learner competes with an adversarial sequ…
Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
Qihao Zhou, Haishan Ye, Luo Luo
This paper considers the distributed convex-concave minimax optimization under the second-order similarity. We propose stochastic variance-reduced optimistic gradient sliding (SVOG…
Optimal Decentralized Composite Optimization for Convex Functions
Haishan Ye, Xiangyu Chang
In this paper, we focus on the decentralized composite optimization for convex functions. Because of advantages such as robust to the network and no communication bottle-neck in th…
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold
Jun Chen, Haishan Ye, Mengmeng Wang +4
The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lowe…
Mirror Natural Evolution Strategies
Haishan Ye
The zeroth-order optimization has been widely used in machine learning applications. However, the theoretical study of the zeroth-order optimization focus on the algorithms which a…
Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
Dachao Lin, Yuze Han, Haishan Ye +1
We study finite-sum distributed optimization problems involving a master node and local nodes under the popular -similarity and -strong convexity conditions. We propose…
Snap-Shot Decentralized Stochastic Gradient Tracking Methods
Haishan Ye, Xiangyu Chang
In decentralized optimization, agents form a network and only communicate with their neighbors, which gives advantages in data ownership, privacy, and scalability. At the same…
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization
Lesi Chen, Haishan Ye, Luo Luo
This paper studies the stochastic nonconvex-strongly-concave minimax optimization over a multi-agent network. We propose an efficient algorithm, called Decentralized Recursive grad…
On the Complexity of Decentralized Smooth Nonconvex Finite-Sum Optimization
Luo Luo, Yunyan Bai, Lesi Chen +2
We study the decentralized optimization problem , where the local function on the -th a…
Decentralized Stochastic Variance Reduced Extragradient Method
Luo Luo, Haishan Ye
This paper studies decentralized convex-concave minimax optimization problems of the form , where is the number…
Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations
Haishan Ye, Dachao Lin, Zhihua Zhang
In this paper, we propose the greedy and random Broyden's method for solving nonlinear equations. Specifically, the greedy method greedily selects the direction to maximize a certa…
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Rui Pan, Haishan Ye, Tong Zhang
Learning rate schedulers have been widely adopted in training deep neural networks. Despite their practical importance, there is a discrepancy between its practice and its theoreti…
Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
Dachao Lin, Haishan Ye, Zhihua Zhang
In this paper, we study the explicit superlinear convergence rates of quasi-Newton methods. We particularly focus on the classical Broyden's method for solving nonlinear equations.…
Explicit Superlinear Convergence Rates of The SR1 Algorithm
Haishan Ye, Dachao Lin, Zhihua Zhang +1
We study the convergence rate of the famous Symmetric Rank-1 (SR1) algorithm which has wide applications in different scenarios. Although it has been extensively investigated, SR1…
Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods
Dachao Lin, Haishan Ye, Zhihua Zhang
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In…
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
Haishan Ye, Tong Zhang
Due to the rapid growth of smart agents such as weakly connected computational nodes and sensors, developing decentralized algorithms that can perform computations on local agents…
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
Haishan Ye, Wei Xiong, Tong Zhang
This paper considers the decentralized composite optimization problem. We propose a novel decentralized variance-reduction proximal-gradient algorithmic framework, called PMGT-VR,…
Revisiting Co-Occurring Directions: Sharper Analysis and Efficient Algorithm for Sparse Matrices
Luo Luo, Cheng Chen, Guangzeng Xie +1
We study the streaming model for approximate matrix multiplication (AMM). We are interested in the scenario that the algorithm can only take one pass over the data with limited mem…
Multi-consensus Decentralized Accelerated Gradient Descent
Haishan Ye, Luo Luo, Ziang Zhou +1
This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory. We…
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
Chaoyang He, Haishan Ye, Li Shen +1
Many recently proposed methods for Neural Architecture Search (NAS) can be formulated as bilevel optimization. For efficient implementation, its solution requires approximations of…