Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data
arXiv:2403.14324 · doi:10.1038/s41377-024-01544-9
Abstract
The integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. This approach also offers the prospect of simplifying hardware requirements/complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function, signal-to-noise ratio, sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field-of-view, depth-of-field, and space-bandwidth product. Here, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span broad applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the future possibilities of this rapidly evolving concept, we hope to motivate our readers to explore novel ways of balancing hardware compromises with compensation via AI.
arXiv comment17 Pages, 4 Figures, 1 Table
References in corpus (22)
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Wide-field, high-resolution Fourier ptychographic microscopy
- DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
- Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
- Deep learning approach to Fourier ptychographic microscopy
- Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning
- Reliable deep-learning-based phase imaging with uncertainty quantification
- Deep learning enhanced mobile-phone microscopy
- Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization
- Self-supervised learning of hologram reconstruction using physics consistency
- Holographic image reconstruction with phase recovery and autofocusing using recurrent neural networks
- Single-shot autofocusing of microscopy images using deep learning
- Deep learning in biomedical optics
- Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopy
- Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
- Recurrent neural network-based volumetric fluorescence microscopy
- eFIN: Enhanced Fourier Imager Network for generalizable autofocusing and pixel super-resolution in holographic imaging
- Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network
- Deep learning-based virtual refocusing of images using an engineered point-spread function
- Increasing a microscope's effective field of view via overlapped imaging and machine learning
- Understanding the limits of remote focusing
- Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems