1.3k citations · 1.7k across the 26 of their papers we have counts for
5 papers · 1 filter
You Look Twice: GaterNet for Dynamic Filter Selection in CNNs
Zhourong Chen, Yang Li, Samy Bengio +1
The concept of conditional computation for deep nets has been proposed previously to improve model performance by selectively using only parts of the model conditioned on the sampl…
Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks
Patrick H. Chen, Si Si, Sanjiv Kumar +2
Neural language models have been widely used in various NLP tasks, including machine translation, next word prediction and conversational agents. However, it is challenging to depl…
Area Attention
Yang Li, Lukasz Kaiser, Samy Bengio +1
Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We p…
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
Patrick H. Chen, Si Si, Yang Li +2
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses. As a case study, a state-of-th…
Nonlinear Online Learning with Adaptive Nyström Approximation
Si Si, Sanjiv Kumar, Yang Li
Use of nonlinear feature maps via kernel approximation has led to success in many online learning tasks. As a popular kernel approximation method, Nyström approximation, has been w…