Deep learning approach to Fourier ptychographic microscopy

Abstract

Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems. The aim of this work is to develop a novel CNN framework to reconstruct video sequence of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided by this large spatial ensemble so as to make predictions in a sequential measurement, without using any additional temporal dataset. Specifically, we show that it is possible to reconstruct high-SBP dynamic cell videos by a CNN trained only on the first FPM dataset captured at the beginning of a time-series experiment. Our CNN approach reconstructs a 12800X10800 pixels phase image using only ~25 seconds, a 50X speedup compared to the model-based FPM algorithm. In addition, the CNN further reduces the required number of images in each time frame by ~6X. Overall, this significantly improves the imaging throughput by reducing both the acquisition and computational times. The proposed CNN is based on the conditional generative adversarial network (cGAN) framework. Additionally, we also exploit transfer learning so that our pre-trained CNN can be further optimized to image other cell types. Our technique demonstrates a promising deep learning approach to continuously monitor large live-cell populations over an extended time and gather useful spatial and temporal information with sub-cellular resolution.

References in corpus (10)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Wide-field, high-resolution Fourier ptychographic microscopy
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Deep Learning Microscopy
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• Experimental robustness of Fourier Ptychography phase retrieval algorithms
• Computational illumination for high-speed in vitro Fourier ptychographic microscopy
• Can Image-Level Labels Replace Pixel-Level Labels for Image Parsing

Cited by in corpus (29)

• DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
• On the use of deep learning for phase recovery
• Reliable deep-learning-based phase imaging with uncertainty quantification
• Learned reconstructions for practical mask-based lensless imaging
• Diffraction tomography with a deep image prior
• Diffractive all-optical computing for quantitative phase imaging
• Deep learning in biomedical optics
• Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement
• Deep learning approach to coherent noise reduction in optical diffraction tomography
• Does deep learning always outperform simple linear regression in optical imaging?
• Deep Iterative Reconstruction for Phase Retrieval
• All-optical complex field imaging using diffractive processors
• Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
• Recurrent neural network-based volumetric fluorescence microscopy
• On the interplay between physical and content priors in deep learning for computational imaging
• Snapshot Ptychography on Array cameras
• Ghost imaging based on Y-net: a dynamic coding and conjugate-decoding approach
• Deep learning phase recovery: data-driven, physics-driven, or combining both?
• High space-bandwidth in quantitative phase imaging using partially spatially coherent optical coherence microscopy and deep neural network
• Deep learning-enabled image quality control in tomographic reconstruction: Robust optical diffraction tomography
• Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network
• Scalable universal tunable virtual-wavevector spatial frequency shift (TVSFS) super-resolution imaging
• Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data
• Integration of Programmable Diffraction with Digital Neural Networks
• Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time
• High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer
• Phase retrieval via Zernike phase contrast microscopy with an untrained neural network
• Quantum process tomography of structured optical gates with convolutional neural networks
• Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks