4 papers
Asymptotic Convergence and Stability of Adaptive Gradient Methods in Smooth Non-convex Optimization
Ruinan Jin, Xiaoyu Wang
Adaptive gradient methods, such as AdaGrad, have become fundamental tools in deep learning. Despite their widespread use, the asymptotic convergence of AdaGrad remains poorly under…
Discontinuous hybrid neural networks for the one-dimensional partial differential equations
Xiaoyu Wang, Long Yuan, Yao Yu
A feedforward neural network, including hidden layers, motivated by nonlinear functions (such as Tanh, ReLU, and Sigmoid functions), exhibits uniform approximation properties in So…
Quantitative Error Bounds for Scaling Limits of Stochastic Iterative Algorithms
Xiaoyu Wang, Mikolaj J. Kasprzak, Jeffrey Negrea +2
Stochastic iterative algorithms, including stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD), are widely utilized for optimization and sampling in…
Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
Ruinan Jin, Xiaoyu Wang, Baoxiang Wang
Adaptive gradient optimizers (AdaGrad), which dynamically adjust the learning rate based on iterative gradients, have emerged as powerful tools in deep learning. These adaptive met…