3.8k citations · 6.6k across the 73 of their papers we have counts for
3 papers · 1 filter
Improving the Gating Mechanism of Recurrent Neural Networks
Albert Gu, Caglar Gulcehre, Tom Le Paine +2
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time. However, their saturation property introd…
Hyperbolic Attention Networks
Caglar Gulcehre, Misha Denil, Mateusz Malinowski +8
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent app…
Model compression via distillation and quantization
Antonio Polino, Razvan Pascanu, Dan Alistarh
Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receivi…