13 citations · 13 across the 1 of their papers we have counts for
8 papers · 1 filter
Big Neural Networks Waste Capacity
Yann N. Dauphin, Yoshua Bengio
This article exposes the failure of some big neural networks to leverage added capacity to reduce underfitting. Past research suggest diminishing returns when increasing the size o…
Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines
Guillaume Desjardins, Razvan Pascanu, Aaron Courville +1
This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm…
A Semantic Matching Energy Function for Learning with Multi-relational Data
Xavier Glorot, Antoine Bordes, Jason Weston +1
Large-scale relational learning becomes crucial for handling the huge amounts of structured data generated daily in many application domains ranging from computational biology or i…
Advances in Optimizing Recurrent Networks
Yoshua Bengio, Nicolas Boulanger-Lewandowski, Razvan Pascanu
After a more than decade-long period of relatively little research activity in the area of recurrent neural networks, several new developments will be reviewed here that have allow…
High-dimensional sequence transduction
Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent
We investigate the problem of transforming an input sequence into a high-dimensional output sequence in order to transcribe polyphonic audio music into symbolic notation. We introd…
Texture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions
Heng Luo, Pierre Luc Carrier, Aaron Courville +1
We apply the spike-and-slab Restricted Boltzmann Machine (ssRBM) to texture modeling. The ssRBM with tiled-convolution weight sharing (TssRBM) achieves or surpasses the state-of-th…