555 citations · 1.7k across the 35 of their papers we have counts for
4 papers · 1 filter
A Cheap Linear Attention Mechanism with Fast Lookups and Fixed-Size Representations
Alexandre de Brébisson, Pascal Vincent
The softmax content-based attention mechanism has proven to be very beneficial in many applications of recurrent neural networks. Nevertheless it suffers from two major computation…
Exact gradient updates in time independent of output size for the spherical loss family
Pascal Vincent, Alexandre de Brébisson, Xavier Bouthillier
An important class of problems involves training deep neural networks with sparse prediction targets of very high dimension D. These occur naturally in e.g. neural language models…
The Z-loss: a shift and scale invariant classification loss belonging to the Spherical Family
Alexandre de Brébisson, Pascal Vincent
Despite being the standard loss function to train multi-class neural networks, the log-softmax has two potential limitations. First, it involves computations that scale linearly wi…
Hierarchical Memory Networks
Sarath Chandar, Sungjin Ahn, Hugo Larochelle +3
Memory networks are neural networks with an explicit memory component that can be both read and written to by the network. The memory is often addressed in a soft way using a softm…