6 citations · 10 across the 10 of their papers we have counts for
10 papers
Dimension-Adaptive Batched Lipschitz Narrowing Without Knowing the Zooming Dimension
Yasong Feng
The Appropriately Combined Edge-length (ACE) sequence in A-BLiN depends on the zooming dimension . This note removes that dependence. The next edge length is selected from the…
Revisiting Stochastic Gradient Descent for Strongly Convex Objectives: Tight Uniform-in-Time Bounds
Kang Chen, Yasong Feng, Tianyu Wang
Stochastic optimization via Stochastic Gradient Descent (SGD) is a fundamental problem in statistics and optimization. This paper revisits Stochastic Gradient Descent (SGD) for str…
Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods
Yasong Feng, Yifan Jiang, Tianyu Wang +1
This work provides a novel convergence analysis for stochastic optimization in terms of stopping times, addressing the practical reality that algorithms are often terminated adapti…
The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective
Yasong Feng, Yifan Jiang, Tianyu Wang +1
We study the stochastic optimization problem from a continuous-time perspective, with a focus on the Stochastic Gradient Descent with Momentum (SGDM) method. We show that the traje…
From Random Search to Bandit Learning in Metric Measure Spaces
Chuying Han, Yasong Feng, Tianyu Wang
Random Search is one of the most widely-used method for Hyperparameter Optimization, and is critical to the success of deep learning models. Despite its astonishing performance, li…
A Lipschitz Bandits Approach for Continuous Hyperparameter Optimization
Yasong Feng, Weijian Luo, Yimin Huang +1
One of the most critical problems in machine learning is HyperParameter Optimization (HPO), since choice of hyperparameters has a significant impact on final model performance. Alt…
Convergence Rates of Stochastic Zeroth-order Gradient Descent for Łojasiewicz Functions
Tianyu Wang, Yasong Feng
We prove convergence rates of Stochastic Zeroth-order Gradient Descent (SZGD) algorithms for Lojasiewicz functions. The SZGD algorithm iterates as \begin{align*} \mathbf{x}_{t+1} =…
Stochastic Zeroth Order Gradient and Hessian Estimators: Variance Reduction and Refined Bias Bounds
Yasong Feng, Tianyu Wang
We study stochastic zeroth order gradient and Hessian estimators for real-valued functions in . We show that, via taking finite difference along random orthogonal dir…
A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective
Yasong Feng, Weiguo Gao
Nesterov's accelerated gradient method (NAG) is widely used in problems with machine learning background including deep learning, and is corresponding to a continuous-time differen…
Lipschitz Bandits with Batched Feedback
Yasong Feng, Zengfeng Huang, Tianyu Wang
In this paper, we study Lipschitz bandit problems with batched feedback, where the expected reward is Lipschitz and the reward observations are communicated to the player in batche…