Universal Denoising without Channel Knowledge

Abstract

Inspired by a classical algorithm for online prediction, we propose a novel denoising scheme which is universal for families of probability distributions both in terms of the source that generates the noiseless signal and in terms of the channel that generates the noisy signal. The new denoising scheme does not rely on assumptions of exact channel knowledge or finite signal alphabets. Our analysis provides an upper bound for the performance gap compared with the Bayes envelope. For the special case in which the signal is an i.i.d. sequence and the family is countable, we characterize the consistency condition under which our scheme approaches the Bayes envelope as the length of the sequence tends to infinity. We also show that in general, approaching the Bayes envelope is not possible for a universal denoising scheme when this consistency condition is not satisfied. Furthermore, we show that, as in the online prediction case, the so-called plug-in approach (which relies on a maximum likelihood estimation of the underlying distribution parameter) does not approach the Bayes envelope in general. We also include numerical evaluations of our scheme for denoising of a Poisson signal.