14.5k citations · 90.1k across the 347 of their papers we have counts for
20 papers · 1 filter
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Ian J. Goodfellow, Mehdi Mirza, Da Xiao +2
Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "fo…
On the number of response regions of deep feed forward networks with piece-wise linear activations
Razvan Pascanu, Guido Montufar, Yoshua Bengio
This paper explores the complexity of deep feedforward networks with linear pre-synaptic couplings and rectified linear activations. This is a contribution to the growing body of w…
How to Construct Deep Recurrent Neural Networks
Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho +1
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear…
On the Challenges of Physical Implementations of RBMs
Vincent Dumoulin, Ian J. Goodfellow, Aaron Courville +1
Restricted Boltzmann machines (RBMs) are powerful machine learning models, but learning and some kinds of inference in the model require sampling-based approximations, which, in cl…
Bounding the Test Log-Likelihood of Generative Models
Yoshua Bengio, Li Yao, Kyunghyun Cho
Several interesting generative learning algorithms involve a complex probability distribution over many random variables, involving intractable normalization constants or latent va…
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu +1
In this paper we propose and investigate a novel nonlinear unit, called unit, for deep neural networks. The proposed unit receives signals from several projections of a…