14.5k citations · 90.1k across the 347 of their papers we have counts for
19 papers · 1 filter
Advances in Optimizing Recurrent Networks
Yoshua Bengio, Nicolas Boulanger-Lewandowski, Razvan Pascanu
After a more than decade-long period of relatively little research activity in the area of recurrent neural networks, several new developments will be reviewed here that have allow…
Joint Training of Deep Boltzmann Machines
Ian Goodfellow, Aaron Courville, Yoshua Bengio
We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at…
High-dimensional sequence transduction
Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent
We investigate the problem of transforming an input sequence into a high-dimensional output sequence in order to transcribe polyphonic audio music into symbolic notation. We introd…
Texture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions
Heng Luo, Pierre Luc Carrier, Aaron Courville +1
We apply the spike-and-slab Restricted Boltzmann Machine (ssRBM) to texture modeling. The ssRBM with tiled-convolution weight sharing (TssRBM) achieves or surpasses the state-of-th…
Theano: new features and speed improvements
Frédéric Bastien, Pascal Lamblin, Razvan Pascanu +6
Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations. In this paper, we prese…
On the difficulty of training Recurrent Neural Networks
Razvan Pascanu, Tomas Mikolov, Yoshua Bengio
There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994). In this pape…