Noise2Self: Blind Denoising by Self-Supervision

Abstract

We propose a general framework for denoising high-dimensional measurements which requires no prior on the signal, no estimate of the noise, and no clean training data. The only assumption is that the noise exhibits statistical independence across different dimensions of the measurement, while the true signal exhibits some correlation. For a broad class of functions ("$\mathcal{J}$-invariant"), it is then possible to estimate the performance of a denoiser from noisy data alone. This allows us to calibrate $\mathcal{J}$-invariant versions of any parameterised denoising algorithm, from the single hyperparameter of a median filter to the millions of weights of a deep neural network. We demonstrate this on natural image and microscopy data, where we exploit noise independence between pixels, and on single-cell gene expression data, where we exploit independence between detections of individual molecules. This framework generalizes recent work on training neural nets from noisy images and on cross-validation for matrix factorization.

10 pages, 6 figures, and supplement

Cited by in corpus (26)

• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images
• Patch2Self: Denoising Diffusion MRI with Self-Supervised Learning
• Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising
• High-Quality Self-Supervised Deep Image Denoising
• Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Images
• NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
• Self-Supervised Training For Low Dose CT Reconstruction
• Adaptive Denoising via GainTuning
• Self-Supervised Fast Adaptation for Denoising via Meta-Learning
• Learning Invariant Representation for Unsupervised Image Restoration
• Self-Supervised Learning based CT Denoising using Pseudo-CT Image Pairs
• Validation and Generalizability of Self-Supervised Image Reconstruction Methods for Undersampled MRI
• Learning to Assimilate in Chaotic Dynamical Systems
• HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models
• Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser
• Joint Demosaicking and Denoising in the Wild: The Case of Training Under Ground Truth Uncertainty
• Restore from Restored: Single-image Inpainting
• View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution
• Blur, Noise, and Compression Robust Generative Adversarial Networks
• Restore from Restored: Single Image Denoising with Pseudo Clean Image
• Learning Model-Blind Temporal Denoisers without Ground Truths
• W2S: Microscopy Data with Joint Denoising and Super-Resolution for Widefield to SIM Mapping
• Legacy Photo Editing with Learned Noise Prior
• Noise Robust Generative Adversarial Networks
• Self-Supervised Inference in State-Space Models