Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks

Patrick H. Chen, Si Si, Sanjiv Kumar +2

Neural language models have been widely used in various NLP tasks, including machine translation, next word prediction and conversational agents. However, it is challenging to depl…