Efficient softmax approximation for GPUs

Edouard Grave, Armand Joulin, Moustapha Cissé +2

We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies. Our approach, called adaptive softmax, circumvents the li…