13 citations · 25 across the 4 of their papers we have counts for
4 papers
Batch Normalization Preconditioning for Neural Network Training
Susanna Lange, Kyle Helfrich, Qiang Ye
Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks…
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory
Kyle Helfrich, Qiang Ye
Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem…
Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform
Kehelwala D. G. Maduranga, Kyle E. Helfrich, Qiang Ye
Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well known difficulty in using RNNs is the \textit{vanishing or explodin…
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform
Kyle Helfrich, Devin Willmott, Qiang Ye
Recurrent Neural Networks (RNNs) are designed to handle sequential data but suffer from vanishing or exploding gradients. Recent work on Unitary Recurrent Neural Networks (uRNNs) h…