Deep Learning Microscopy

Abstract

We demonstrate that a deep neural network can significantly improve optical microscopy, enhancing its spatial resolution over a large field-of-view and depth-of-field. After its training, the only input to this network is an image acquired using a regular optical microscope, without any changes to its design. We blindly tested this deep learning approach using various tissue samples that are imaged with low-resolution and wide-field systems, where the network rapidly outputs an image with remarkably better resolution, matching the performance of higher numerical aperture lenses, also significantly surpassing their limited field-of-view and depth-of-field. These results are transformative for various fields that use microscopy tools, including e.g., life sciences, where optical microscopy is considered as one of the most widely used and deployed techniques. Beyond such applications, our presented approach is broadly applicable to other imaging modalities, also spanning different parts of the electromagnetic spectrum, and can be used to design computational imagers that get better and better as they continue to image specimen and establish new transformations among different modes of imaging.

References in corpus (5)

• Deep Learning in Neural Networks: An Overview
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Deep Bilateral Learning for Real-Time Image Enhancement

Cited by in corpus (86)

• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
• Deep learning in nano-photonics: inverse design and beyond
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• Learning to see through multimode fibers
• Intelligent Nanophotonics: Merging Photonics and Artificial Intelligence at the Nanoscale
• Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• Deep learning approach to Fourier ptychographic microscopy
• On the use of deep learning for phase recovery
• Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning
• Terahertz Pulse Shaping Using Diffractive Surfaces
• Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
• Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue
• Deep learning enhanced mobile-phone microscopy
• Misalignment Resilient Diffractive Optical Networks
• Ensemble learning of diffractive optical networks
• Resolution enhancement in scanning electron microscopy using deep learning
• Artificial Neural Network with Physical Dynamic Metasurface Layer for Optimal Sensing
• Deep learning-based super-resolution in coherent imaging systems
• Light scattering control in transmission and reflection with neural networks
• Self-supervised learning of hologram reconstruction using physics consistency
• Subwavelength resolution Fourier ptychography with hemispherical digital condensers
• Plasmonic colours predicted by deep learning
• Deep Null Space Learning for Inverse Problems: Convergence Analysis and Rates
• Single-shot autofocusing of microscopy images using deep learning
• Deep learning in biomedical optics
• Deep learning enables high-throughput analysis of particle-aggregation-based bio-sensors imaged using holography
• High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks
• Deep learning approach to coherent noise reduction in optical diffraction tomography
• Does deep learning always outperform simple linear regression in optical imaging?
• Practical sensorless aberration estimation for 3D microscopy with deep learning
• Machine learning for faster and smarter fluorescence lifetime imaging microscopy
• A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors
• Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
• Ten Quick Tips for Deep Learning in Biology
• Recurrent neural network-based volumetric fluorescence microscopy
• Adaptive 3D descattering with a dynamic synthesis network
• Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks
• On the interplay between physical and content priors in deep learning for computational imaging
• Far-field subwavelength acoustic imaging by deep learning
• Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter
• Spatial images from temporal data
• Exploring particle dynamics during self-organization processes via rotationally invariant latent representations
• Spectral pre-modulation of training examples enhances the spatial resolution of the Phase Extraction Neural Network (PhENN)
• Pixel super-resolved virtual staining of label-free tissue using diffusion models
• Limitations of Deep Learning for Inverse Problems on Digital Hardware
• Machine learning reveals complex behaviours in optically trapped particles
• Optimal Physical Preprocessing for Example-Based Super-Resolution
• Imaging through glass diffusers using densely connected convolutional networks
• Inverse Design of All-dielectric Metasurfaces with Bound States in the Continuum
• Deep learning-based virtual refocusing of images using an engineered point-spread function
• Increasing a microscope's effective field of view via overlapped imaging and machine learning
• Single-Pixel Pattern Recognition with Coherent Nonlinear Optics
• Deep Learning Enhanced Quantum Holography with Undetected Photons
• Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery
• Virtual Staining of Label-Free Tissue in Imaging Mass Spectrometry
• Acoustic Structure Inverse Design and Optimization Using Deep Learning
• Virtual organelle self-coding for fluorescence imaging via adversarial learning
• Toward a Thinking Microscope: Deep Learning in Optical Microscopy and Image Reconstruction
• Faster calculations of optical trapping using neural networks trained by T-matrix data: an application to micro and nanoplastics
• Autofocusing technologies for whole slide imaging and automated microscopy
• High-throughput, high-resolution registration-free generated adversarial network microscopy
• Identification of particle mixtures using machine-learning-assisted laser diffraction analysis
• Learning an optimal PSF-pair for ultra-dense 3D localization microscopy
• A New Deep Learning Method for Image Deblurring in Optical Microscopic Systems
• Analog-to-digital conversion revolutionized by deep learning
• Deep learning-based phase control method for coherent beam combining and its application in generating orbital angular momentum beams
• Competing Neural Networks for Robust Control of Nonlinear Systems
• Far-field intensity signature of sub-wavelength microscopic objects
• Deep learning based high-resolution incoherent x-ray imaging with a single-pixel detector
• BlurryScope enables compact, cost-effective scanning microscopy for HER2 scoring using deep learning on blurry images
• Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks
• Deep learning scheme for recovery of broadband microwave photonic receiving systems in transceivers without expert knowledge and system priors
• Learning the geometry of wave-based imaging
• From Stars to Molecules: AI Guided Device-Agnostic Super-Resolution Imaging
• Quantifying the effect of image compression on supervised learning applications in optical microscopy
• Alignment of electron optical beam shaping elements using a convolutional neural network
• Pixel Super-Resolved Fluorescence Lifetime Imaging Using Deep Learning
• Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments
• Spectral splitting and concentration of broadband light using neural networks
• Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging
• Variational framework for partially-measured physical system control: examples of vision neuroscience and optical random media