14.5k citations · 90.1k across the 347 of their papers we have counts for
29 papers · 1 filter
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho +1
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism,…
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho +1
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurre…
Difference Target Propagation
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer +1
Back-propagation has been the workhorse of recent successes of deep learning but it relies on infinitesimal effects (partial derivatives) in order to perform credit assignment. Thi…
ADASECANT: Robust Adaptive Secant Method for Stochastic Gradient
Caglar Gulcehre, Marcin Moczulski, Yoshua Bengio
Stochastic gradient algorithms have been the main focus of large-scale learning problems and they led to important successes in machine learning. The convergence of SGD depends on…
Training deep neural networks with low precision multiplications
Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David
Multipliers are the most space and power-hungry arithmetic operators of the digital implementation of deep neural networks. We train a set of state-of-the-art neural networks (Maxo…
FitNets: Hints for Thin Deep Nets
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou +3
While depth tends to improve network performances, it also makes gradient-based training more difficult since deeper networks tend to be more non-linear. The recently proposed know…