5 papers
"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihood
Peiyu Yu, Dinghuai Zhang, Hengzhi He +10
Noise Contrastive Estimation (NCE) has fueled major breakthroughs in representation learning and generative modeling. Yet a long-standing challenge remains: accurately estimating r…
On Conformal Isometry of Grid Cells: Learning Distance-Preserving Position Embedding
Dehong Xu, Ruiqi Gao, Wen-Hao Zhang +2
This paper investigates the conformal isometry hypothesis as a potential explanation for the hexagonal periodic patterns in grid cell response maps. We posit that grid cell activit…
EM Distillation for One-step Diffusion Models
Sirui Xie, Zhisheng Xiao, Diederik P Kingma +6
While diffusion models can learn complex distributions, sampling requires a computationally expensive iterative process. Existing distillation methods enable efficient sampling, bu…
Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood
Yaxuan Zhu, Jianwen Xie, Yingnian Wu +1
Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other…
Think Twice Before You Act: Improving Inverse Problem Solving With MCMC
Yaxuan Zhu, Zehao Dou, Haoxin Zheng +3
Recent studies demonstrate that diffusion models can serve as a strong prior for solving inverse problems. A prominent example is Diffusion Posterior Sampling (DPS), which approxim…