The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translation

Evgeniia Tokarchuk, Vlad Niculae

Continuous-output neural machine translation (CoNMT) replaces the discrete next-word prediction problem with an embedding prediction. The semantic structure of the target embedding…