9 citations · 51 across the 24 of their papers we have counts for
6 papers · 1 filter
Escape saddle points faster on manifolds via perturbed Riemannian stochastic recursive gradient
Andi Han, Junbin Gao
In this paper, we propose a variant of Riemannian stochastic recursive gradient method that can achieve second-order convergence guarantee and escape saddle points using simple per…
Regularized Flexible Activation Function Combinations for Deep Neural Networks
Renlong Jie, Junbin Gao, Andrey Vasnev +1
Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning t…
Riemannian stochastic recursive momentum method for non-convex optimization
Andi Han, Junbin Gao
We propose a stochastic recursive momentum method for Riemannian non-convex optimization that achieves a near-optimal complexity of to find -approx…
Variance reduction for Riemannian non-convex optimization with batch size adaptation
Andi Han, Junbin Gao
Variance reduction techniques are popular in accelerating gradient descent and stochastic gradient descent for optimization problems defined on both Euclidean space and Riemannian…
MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning
Xuebin Zheng, Bingxin Zhou, Ming Li +2
Graph Neural Networks (GNNs) have recently caught great attention and achieved significant progress in graph-level applications. In this paper, we propose a framework for graph neu…
On the Trend-corrected Variant of Adaptive Stochastic Optimization Methods
Bingxin Zhou, Xuebin Zheng, Junbin Gao
Adam-type optimizers, as a class of adaptive moment estimation methods with the exponential moving average scheme, have been successfully used in many applications of deep learning…