Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network

Abstract

Deep learning-based methods in computational microscopy have been shown to be powerful but in general face some challenges due to limited generalization to new types of samples and requirements for large and diverse training data. Here, we demonstrate a few-shot transfer learning method that helps a holographic image reconstruction deep neural network rapidly generalize to new types of samples using small datasets. We pre-trained a convolutional recurrent neural network on a large dataset with diverse types of samples, which serves as the backbone model. By fixing the recurrent blocks and transferring the rest of the convolutional blocks of the pre-trained model, we reduced the number of trainable parameters by ~90% compared with standard transfer learning, while achieving equivalent generalization. We validated the effectiveness of this approach by successfully generalizing to new types of samples using small holographic datasets for training, and achieved (i) ~2.5-fold convergence speed acceleration, (ii) ~20% computation time reduction per epoch, and (iii) improved reconstruction performance over baseline network models trained from scratch. This few-shot transfer learning approach can potentially be applied in other microscopic imaging methods, helping to generalize to new types of samples without the need for extensive training time and data.

arXiv comment10 Pages, 3 Figures

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
• Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
• Deep learning-based transformation of the H&E stain into special stains
• Deep learning approach to Fourier ptychographic microscopy
• Low Photon Count Phase Retrieval Using Deep Learning
• Cross-modality deep learning brings bright-field microscopy contrast to holography
• Holographic image reconstruction with phase recovery and autofocusing using recurrent neural networks
• Recurrent neural network-based volumetric fluorescence microscopy
• Deep learning-based virtual refocusing of images using an engineered point-spread function

Cited by in corpus (4)

• Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization
• Self-supervised learning of hologram reconstruction using physics consistency
• Virtual stain transfer in histology via cascaded deep neural networks
• Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data