7.8k citations · 21.1k across the 144 of their papers we have counts for
6 papers · 1 filter
On the number of response regions of deep feed forward networks with piece-wise linear activations
Razvan Pascanu, Guido Montufar, Yoshua Bengio
This paper explores the complexity of deep feedforward networks with linear pre-synaptic couplings and rectified linear activations. This is a contribution to the growing body of w…
How to Construct Deep Recurrent Neural Networks
Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho +1
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear…
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu +1
In this paper we propose and investigate a novel nonlinear unit, called unit, for deep neural networks. The proposed unit receives signals from several projections of a…
Pylearn2: a machine learning research library
Ian J. Goodfellow, David Warde-Farley, Pascal Lamblin +6
Pylearn2 is a machine learning research library. This does not just mean that it is a collection of machine learning algorithms that share a common API; it means that it has been d…
Revisiting Natural Gradient for Deep Networks
Razvan Pascanu, Yoshua Bengio
We evaluate natural gradient, an algorithm originally proposed in Amari (1997), for learning deep models. The contributions of this paper are as follows. We show the connection bet…
Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines
Guillaume Desjardins, Razvan Pascanu, Aaron Courville +1
This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm…