1.3k citations · 2.9k across the 32 of their papers we have counts for
Showing cs.NEShow all
2 papers · 1 filter
cs.NE2016★ 167 cited
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
David Krueger, Tegan Maharaj, János Kramár +7
We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout…
cs.NE2015★ 27 cited
Regularizing RNNs by Stabilizing Activations
David Krueger, Roland Memisevic
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regul…