Adaptive Denoising via GainTuning

Abstract

Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. These models achieve the current state of the art, but they have difficulties generalizing when applied to data that deviate from the training distribution. Recent work has shown that it is possible to train denoisers on a single noisy image. These models adapt to the features of the test image, but their performance is limited by the small amount of information used to train them. Here we propose "GainTuning", in which CNN models pre-trained on large datasets are adaptively and selectively adjusted for individual test images. To avoid overfitting, GainTuning optimizes a single multiplicative scaling parameter (the "Gain") of each channel in the convolutional layers of the CNN. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set. These adaptive improvements are even more substantial for test images differing systematically from the training data, either in noise level or image type. We illustrate the potential of adaptive denoising in a scientific application, in which a CNN is trained on synthetic data, and tested on real transmission-electron-microscope images. In contrast to the existing methodology, GainTuning is able to faithfully reconstruct the structure of catalytic nanoparticles from these data at extremely low signal-to-noise ratios.

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• How transferable are features in deep neural networks?
• Modulating early visual processing by language
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Improving robustness against common corruptions by covariate shift adaptation
• Noise2Self: Blind Denoising by Self-Supervision
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• Developing and Evaluating Deep Neural Network-based Denoising for Nanoparticle TEM Images with Ultra-low Signal-to-Noise
• Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising
• Be Like Water: Robustness to Extraneous Variables Via Adaptive Feature Normalization
• Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy