42 citations · 163 across the 36 of their papers we have counts for
14 papers · 1 filter
Logarithmic High-Probability Regret for Online Convex Optimization with Two-Point Bandit Feedback
Haishan Ye
We study online convex optimization (OCO) with two-point bandit feedback against a non-anticipating adaptive adversary. In this setting, a learner competes with an adversarial sequ…
Mirror Natural Evolution Strategies
Haishan Ye
The zeroth-order optimization has been widely used in machine learning applications. However, the theoretical study of the zeroth-order optimization focus on the algorithms which a…
Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
Dachao Lin, Yuze Han, Haishan Ye +1
We study finite-sum distributed optimization problems involving a master node and local nodes under the popular -similarity and -strong convexity conditions. We propose…
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization
Lesi Chen, Haishan Ye, Luo Luo
This paper studies the stochastic nonconvex-strongly-concave minimax optimization over a multi-agent network. We propose an efficient algorithm, called Decentralized Recursive grad…
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Rui Pan, Haishan Ye, Tong Zhang
Learning rate schedulers have been widely adopted in training deep neural networks. Despite their practical importance, there is a discrepancy between its practice and its theoreti…
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
Haishan Ye, Tong Zhang
Due to the rapid growth of smart agents such as weakly connected computational nodes and sensors, developing decentralized algorithms that can perform computations on local agents…
Revisiting Co-Occurring Directions: Sharper Analysis and Efficient Algorithm for Sparse Matrices
Luo Luo, Cheng Chen, Guangzeng Xie +1
We study the streaming model for approximate matrix multiplication (AMM). We are interested in the scenario that the algorithm can only take one pass over the data with limited mem…
Multi-consensus Decentralized Accelerated Gradient Descent
Haishan Ye, Luo Luo, Ziang Zhou +1
This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory. We…
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
Chaoyang He, Haishan Ye, Li Shen +1
Many recently proposed methods for Neural Architecture Search (NAS) can be formulated as bilevel optimization. For efficient implementation, its solution requires approximations of…
Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problems
Luo Luo, Haishan Ye, Zhichao Huang +1
We consider nonconvex-concave minimax optimization problems of the form , where is strongly-concave in but p…
Fast Generalized Matrix Regression with Applications in Machine Learning
Haishan Ye, Shusen Wang, Zhihua Zhang +1
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation s…
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Haishan Ye, Zhichao Huang, Cong Fang +2
Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based imag…
Nesterov's Acceleration For Approximate Newton
Haishan Ye, Zhihua Zhang
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. Ho…
Nestrov's Acceleration For Second Order Method
Haishan Ye, Zhihua Zhang
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. Ho…