3.8k citations · 4.9k across the 5 of their papers we have counts for
5 papers
Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines
Guillaume Desjardins, Razvan Pascanu, Aaron Courville +1
This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm…
Advances in Optimizing Recurrent Networks
Yoshua Bengio, Nicolas Boulanger-Lewandowski, Razvan Pascanu
After a more than decade-long period of relatively little research activity in the area of recurrent neural networks, several new developments will be reviewed here that have allow…
Theano: new features and speed improvements
Frédéric Bastien, Pascal Lamblin, Razvan Pascanu +6
Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations. In this paper, we prese…
On the difficulty of training Recurrent Neural Networks
Razvan Pascanu, Tomas Mikolov, Yoshua Bengio
There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994). In this pape…
Deep Self-Taught Learning for Handwritten Character Recognition
Frédéric Bastien, Yoshua Bengio, Arnaud Bergeron +14
Recent theoretical and empirical work in statistical machine learning has demonstrated the importance of learning algorithms for deep architectures, i.e., function classes obtained…