Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization

Abstract

Deep learning-based image reconstruction methods have achieved remarkable success in phase recovery and holographic imaging. However, the generalization of their image reconstruction performance to new types of samples never seen by the network remains a challenge. Here we introduce a deep learning framework, termed Fourier Imager Network (FIN), that can perform end-to-end phase recovery and image reconstruction from raw holograms of new types of samples, exhibiting unprecedented success in external generalization. FIN architecture is based on spatial Fourier transform modules that process the spatial frequencies of its inputs using learnable filters and a global receptive field. Compared with existing convolutional deep neural networks used for hologram reconstruction, FIN exhibits superior generalization to new types of samples, while also being much faster in its image inference speed, completing the hologram reconstruction task in ~0.04 s per 1 mm^2 of the sample area. We experimentally validated the performance of FIN by training it using human lung tissue samples and blindly testing it on human prostate, salivary gland tissue and Pap smear samples, proving its superior external generalization and image reconstruction speed. Beyond holographic microscopy and quantitative phase imaging, FIN and the underlying neural network architecture might open up various new opportunities to design broadly generalizable deep learning models in computational imaging and machine vision fields.

19 Pages, 5 Figures, 1 Table

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Generative Adversarial Networks
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• A unified evaluation of iterative projection algorithms for phase retrieval
• Deep learning-based color holographic microscopy
• Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network

Cited by in corpus (12)

• On the use of deep learning for phase recovery
• Self-supervised learning of hologram reconstruction using physics consistency
• Multiplane Quantitative Phase Imaging Using a Wavelength-Multiplexed Diffractive Optical Processor
• All-optical complex field imaging using diffractive processors
• Physics-driven universal twin-image removal network for digital in-line holographic microscopy
• Deep learning phase recovery: data-driven, physics-driven, or combining both?
• Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network
• eFIN: Enhanced Fourier Imager Network for generalizable autofocusing and pixel super-resolution in holographic imaging
• Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data
• OAH-Net: A Deep Neural Network for Hologram Reconstruction of Off-axis Digital Holographic Microscope
• BlurryScope enables compact, cost-effective scanning microscopy for HER2 scoring using deep learning on blurry images
• Snapshot 3D image projection using a diffractive decoder