All-optical image denoising using a diffractive visual processor

Abstract

Image denoising, one of the essential inverse problems, targets to remove noise/artifacts from input images. In general, digital image denoising algorithms, executed on computers, present latency due to several iterations implemented in, e.g., graphics processing units (GPUs). While deep learning-enabled methods can operate non-iteratively, they also introduce latency and impose a significant computational burden, leading to increased power consumption. Here, we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images - implemented at the speed of light propagation within a thin diffractive visual processor. This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features, causing them to miss the output image Field-of-View (FoV) while retaining the object features of interest. Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of ~30-40%. We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum. Owing to their speed, power-efficiency, and minimal computational overhead, all-optical diffractive denoisers can be transformative for various image display and projection systems, including, e.g., holographic displays.

arXiv comment21 Pages, 7 Figures

References in corpus (29)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Deep Bilateral Learning for Real-Time Image Enhancement
• A Review of Convolutional Neural Networks for Inverse Problems in Imaging
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• GhostNetV2: Enhance Cheap Operation with Long-Range Attention
• Image sensing with multilayer, nonlinear optical neural networks
• Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks
• Compute and Energy Consumption Trends in Deep Learning Inference
• Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
• Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
• All-Optical Information Processing Capacity of Diffractive Surfaces
• Misalignment Resilient Diffractive Optical Networks
• Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Network
• Snapshot Multispectral Imaging Using a Diffractive Optical Network
• Diffractive all-optical computing for quantitative phase imaging
• Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network
• To image, or not to image: Class-specific diffractive cameras with all-optical erasure of undesired objects
• Scale-, shift- and rotation-invariant diffractive optical networks
• Universal Linear Intensity Transformations Using Spatially-Incoherent Diffractive Processors
• Super-resolution image display using diffractive decoders
• Data class-specific all-optical transformations and encryption
• Unidirectional Imaging using Deep Learning-Designed Materials
• Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network
• Learning Diffractive Optical Communication Around Arbitrary Opaque Occlusions
• Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network
• Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding

Cited by in corpus (7)

• Optical Generative Models
• High-performance real-world optical computing trained by in situ gradient-based model-free optimization
• Meta-operators for all-optical image processing
• Broadband Unidirectional Visible Imaging Using Wafer-Scale Nano-Fabrication of Multi-Layer Diffractive Optical Processors
• Engineering nonlinear activation functions for all-optical neural networks via quantum interference
• Compute-first optical detection for noise-resilient visual perception
• Snapshot 3D image projection using a diffractive decoder