10 citations · 17 across the 5 of their papers we have counts for
Showing 2016Show all
2 papers · 1 filter
cs.NE2016★ 1 cited
Sampling-based Gradient Regularization for Capturing Long-Term Dependencies in Recurrent Neural Networks
Artem Chernodub, Dimitri Nowicki
Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of…
cs.NE2016★ 10 cited
Norm-preserving Orthogonal Permutation Linear Unit Activation Functions (OPLU)
Artem Chernodub, Dimitri Nowicki
We propose a novel activation function that implements piece-wise orthogonal non-linear mappings based on permutations. It is straightforward to implement, and very computationally…