105 citations · 533 across the 41 of their papers we have counts for
5 papers · 1 filter
Hessian-Free High-Resolution Nesterov Acceleration for Sampling
Ruilin Li, Hongyuan Zha, Molei Tao
Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep…
Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients
Ruilin Li, Xin Wang, Hongyuan Zha +1
Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing…
Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective
Kaixuan Huang, Yuqing Wang, Molei Tao +1
Deep residual networks (ResNets) have demonstrated better generalization performance than deep feedforward networks (FFNets). However, the theory behind such a phenomenon is still…
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function
Lingkai Kong, Molei Tao
This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors. In particular, it shows…
Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems
Molei Tao, Tomoki Ohsawa
The article considers smooth optimization of functions on Lie groups. By generalizing NAG variational principle in vector space (Wibisono et al., 2016) to Lie groups, continuous Li…