14.1k citations · 34.6k across the 53 of their papers we have counts for
Showing 2015Show all
2 papers · 1 filter
cs.NE2015★ 554 cited
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton
Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients. To overcome this difficulty, researchers have developed sophisticated o…
stat.ML2015★ 14.1k cited
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, Jeff Dean
A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfo…