Self-Normalized Importance Sampling for Neural Language Modeling

Zijian Yang, Yingbo Gao, Alexander Gerstenberger +3

To mitigate the problem of having to traverse over the full vocabulary in the softmax normalization of a neural language model, sampling-based training criteria are proposed and in…