The Optimal Noise in Noise-Contrastive Learning Is Not What You Think

Omar Chehab, Alexandre Gramfort, Aapo Hyvarinen

Learning a parametric model of a data distribution is a well-known statistical problem that has seen renewed interest as it is brought to scale in deep learning. Framing the proble…