most citedAdaptive-saturated RNN: Remember more with less instability

1 citations · 1 across the 1 of their papers we have counts for

collaborators

1 paper

cs.LG2023★ 1 cited

Adaptive-saturated RNN: Remember more with less instability

Khoi Minh Nguyen-Duy, Quang Pham, Binh T. Nguyen

Orthogonal parameterization is a compelling solution to the vanishing gradient problem (VGP) in recurrent neural networks (RNNs). With orthogonal parameters and non-saturated activ…