Recurrent neural network-based volumetric fluorescence microscopy

Abstract

Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. Elegans and nanobead samples, Recurrent-MZ is demonstrated to increase the depth-of-field of a 63x/1.4NA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.

arXiv comment17 pages, 7 figures

References in corpus (15)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Methods for Interpreting and Understanding Deep Neural Networks
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• Deep Learning Microscopy
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
• Deep learning approach to Fourier ptychographic microscopy
• Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning
• Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
• Single-shot autofocusing of microscopy images using deep learning
• Compressive three-dimensional super-resolution microscopy with speckle-saturated fluorescence excitation
• Addressing systematic errors in axial distance measurements in single-emitter localization microscopy
• Continuous focal translation enhances rate of point-scan volumetric microscopy

Cited by in corpus (9)

• Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization
• Self-supervised learning of hologram reconstruction using physics consistency
• Holographic image reconstruction with phase recovery and autofocusing using recurrent neural networks
• Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
• Adaptive 3D descattering with a dynamic synthesis network
• Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks
• Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network
• Deep learning-based virtual refocusing of images using an engineered point-spread function
• Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data