873 citations · 5.4k across the 120 of their papers we have counts for
7 papers · 2 filters
From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization
Qiuwei Li, Daniel McKenzie, Wotao Yin
The standard simplex in R^n, also known as the probability simplex, is the set of nonnegative vectors whose entries sum up to 1. They frequently appear as constraints in optimizati…
Curvature-Aware Derivative-Free Optimization
Bumsu Kim, HanQin Cai, Daniel McKenzie +1
The paper discusses derivative-free optimization (DFO), which involves minimizing a function without access to gradients or directional derivatives, only function evaluations. Clas…
Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator
Zhishuai Guo, Yi Xu, Wotao Yin +2
Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. T…
Learning to Optimize: A Primer and A Benchmark
Tianlong Chen, Xiaohan Chen, Wuyang Chen +4
Learning to optimize (L2O) is an emerging approach that leverages machine learning to develop optimization methods, aiming at reducing the laborious iterations of hand engineering.…
A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization
HanQin Cai, Yuchen Lou, Daniel McKenzie +1
We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the deci…
A Single-Timescale Method for Stochastic Bilevel Optimization
Tianyi Chen, Yuejiao Sun, Quan Xiao +1
Stochastic bilevel optimization generalizes the classic stochastic optimization from the minimization of a single objective to the minimization of an objective function that depend…