MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy

Davis Yoshida, Kartik Goyal, Kevin Gimpel

It has been widely observed that exact or approximate MAP (mode-seeking) decoding from natural language generation (NLG) models consistently leads to degenerate outputs (Holtzman e…