Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Understanding Adam Requires Better Rotation Dependent Assumptions
Tianyue H. Zhang, Lucas Maes, Alan Milligan +5
Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This paper investigates Adam's sensitivity t…
cs.LG2024
On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization
Motahareh Sohrabi, Juan Ramirez, Tianyue H. Zhang +2
Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problems are solved via their min-max Lagrangia…
cs.LG2024
Efficient and Adaptive Posterior Sampling Algorithms for Bandits
Bingshan Hu, Zhiming Huang, Tianyue H. Zhang +2
We study Thompson Sampling-based algorithms for stochastic bandits with bounded rewards. As the existing problem-dependent regret bound for Thompson Sampling with Gaussian priors […