10.8k citations · 17.5k across the 28 of their papers we have counts for
6 papers · 1 filter
Improving the Gating Mechanism of Recurrent Neural Networks
Albert Gu, Caglar Gulcehre, Tom Le Paine +2
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time. However, their saturation property introd…
Hyperbolic Attention Networks
Caglar Gulcehre, Misha Denil, Mateusz Malinowski +8
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent app…
Gated Feedback Recurrent Neural Networks
Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho +1
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recu…
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho +1
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism,…
How to Construct Deep Recurrent Neural Networks
Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho +1
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear…
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu +1
In this paper we propose and investigate a novel nonlinear unit, called unit, for deep neural networks. The proposed unit receives signals from several projections of a…