Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula

Abstract

We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise $Z$ corrupts a signal $X$, yielding the observation $Y=X+\sigma Z$ with known $\sigma \in (0,1)$. We propose \emph{universal} denoisers, agnostic to both signal and noise distributions, that recover the signal distribution $P_X$ from $P_Y$. When the focus is on distributional recovery of $P_X$ rather than on individual realizations of $X$, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, which achieves $O({\sigma }^2)$ accuracy. They shrink $P_Y$ toward $P_X$ with $O({\sigma }^4)$ and $O({\sigma }^6)$ accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge--Ampère equation with higher-order accuracy and can be implemented efficiently via score matching. Let $q$ denote the density of $P_Y$. For distributional denoising, we propose replacing the Bayes-optimal denoiser, $${\mathbf{T}}^{\ast }(y)=y+{\sigma }^2\nabla \log q(y),$$ with denoisers exhibiting less-aggressive distributional shrinkage, $${\mathbf{T}}_1(y)=y+\frac{{\sigma }^2}{2}\nabla \log q(y),$$ $${\mathbf{T}}_2(y)=y+\frac{{\sigma }^2}{2}\nabla \log q(y)-\frac{{\sigma }^4}{8}\nabla \left(\frac{1}{2}\Vert \nabla \log q(y){\Vert }^2+\nabla \cdot \nabla \log q(y)\right).$$

27 pages, 5 figures