29 citations · 73 across the 13 of their papers we have counts for
13 papers
Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum
Mengqi Hu, Yifei Lou, Bao Wang +3
This paper applies an idea of adaptive momentum for the nonlinear conjugate gradient to accelerate optimization problems in sparse recovery. Specifically, we consider two types of…
AUTM Flow: Atomic Unrestricted Time Machine for Monotonic Normalizing Flows
Difeng Cai, Yuliang Ji, Huan He +2
Nonlinear monotone transformations are used extensively in normalizing flows to construct invertible triangular mappings from simple distributions to complex ones. In existing lite…
Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum
Bao Wang, Qiang Ye
Momentum plays a crucial role in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, th…
Adaptive Weighted Discriminator for Training Generative Adversarial Networks
Vasily Zadorozhnyy, Qiang Cheng, Qiang Ye
Generative adversarial network (GAN) has become one of the most important neural network models for classical unsupervised machine learning. A variety of discriminator loss functio…
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory
Kyle Helfrich, Qiang Ye
Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem…
On regularization for a convolutional kernel in neural networks
Peichang Guo, Qiang Ye
Convolutional neural network is an important model in deep learning. To avoid exploding/vanishing gradient problems and to improve the generalizability of a neural network, it is d…