Counting papers with at most 25 authors, so collaboration author lists do not stand in for the people here.
- Centre National de la Recherche ScientifiqueFR2k papers
- University of British ColumbiaCA1.9k papers
- McGill UniversityCA1.9k papers
- University of VictoriaCA1.8k papers
- University of California, Santa CruzUS1.7k papers
- Rutherford Appleton LaboratoryGB1.7k papers
- University of ManchesterGB1.7k papers
- Queen Mary University of LondonGB1.7k papers
- University of PisaIT1.6k papers
- University of EdinburghGB1.6k papers
- University of BirminghamGB1.6k papers
- Heidelberg UniversityDE1.6k papers
11 papers · 2 filters
Advances in Optimizing Recurrent Networks
Yoshua Bengio, Nicolas Boulanger-Lewandowski, Razvan Pascanu
After a more than decade-long period of relatively little research activity in the area of recurrent neural networks, several new developments will be reviewed here that have allow…
High-dimensional sequence transduction
Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent
We investigate the problem of transforming an input sequence into a high-dimensional output sequence in order to transcribe polyphonic audio music into symbolic notation. We introd…
Texture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions
Heng Luo, Pierre Luc Carrier, Aaron Courville +1
We apply the spike-and-slab Restricted Boltzmann Machine (ssRBM) to texture modeling. The ssRBM with tiled-convolution weight sharing (TssRBM) achieves or surpasses the state-of-th…
On the difficulty of training Recurrent Neural Networks
Razvan Pascanu, Tomas Mikolov, Yoshua Bengio
There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994). In this pape…
What Regularized Auto-Encoders Learn from the Data Generating Distribution
Guillaume Alain, Yoshua Bengio
What do auto-encoders learn about the underlying data generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold st…
Better Mixing via Deep Representations
Yoshua Bengio, Grégoire Mesnil, Yann Dauphin +1
It has previously been hypothesized, and supported with some experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the und…
A Generative Process for Sampling Contractive Auto-Encoders
Salah Rifai, Yoshua Bengio, Yann Dauphin +1
The contractive auto-encoder learns a representation of the input data that captures the local manifold structure around each data point, through the leading singular vectors of th…
Large-Scale Feature Learning With Spike-and-Slab Sparse Coding
Ian Goodfellow, Aaron Courville, Yoshua Bengio
We consider the problem of object recognition with a large number of classes. In order to overcome the low amount of labeled examples available in this setting, we introduce a new…
Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent
We investigate the problem of modeling symbolic sequences of polyphonic music in a completely general piano-roll representation. We introduce a probabilistic model based on distrib…
Representation Learning: A Review and New Perspectives
Yoshua Bengio, Aaron Courville, Pascal Vincent
The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or…
Practical recommendations for gradient-based training of deep architectures
Yoshua Bengio
Learning algorithms related to artificial neural networks and in particular for Deep Learning may seem to involve many bells and whistles, called hyper-parameters. This chapter is…