555 citations · 1.7k across the 35 of their papers we have counts for
Showing 2016 · cs.LGShow all
2 papers · 2 filters
cs.LG2016★ 6 cited
A Cheap Linear Attention Mechanism with Fast Lookups and Fixed-Size Representations
Alexandre de Brébisson, Pascal Vincent
The softmax content-based attention mechanism has proven to be very beneficial in many applications of recurrent neural networks. Nevertheless it suffers from two major computation…
cs.LG2016★ 5 cited
The Z-loss: a shift and scale invariant classification loss belonging to the Spherical Family
Alexandre de Brébisson, Pascal Vincent
Despite being the standard loss function to train multi-class neural networks, the log-softmax has two potential limitations. First, it involves computations that scale linearly wi…