Deep-STORM: super-resolution single-molecule microscopy by deep learning

Abstract

We present an ultra-fast, precise, parameter-free method, which we term Deep-STORM, for obtaining super-resolution images from stochastically-blinking emitters, such as fluorescent molecules used for localization microscopy. Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated. The method achieves state-of-the-art resolution under challenging signal-to-noise conditions and high emitter densities, and is significantly faster than existing approaches. Additionally, no prior information on the shape of the underlying structure is required, making the method applicable to any blinking data-set. We validate our approach by super-resolution image reconstruction of simulated and experimentally obtained data.

7 pages, added code download reference and DOI for the journal version

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• Deep Learning Microscopy
• Learning Deconvolution Network for Semantic Segmentation
• Deep Networks for Image Super-Resolution with Sparse Prior

Cited by in corpus (75)

• Deep learning in nano-photonics: inverse design and beyond
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
• Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• Deep learning approach to Fourier ptychographic microscopy
• Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning
• Convolutional neural networks automate detection for tracking of submicron scale particles in 2D and 3D
• Terahertz Pulse Shaping Using Diffractive Surfaces
• Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
• Misalignment Resilient Diffractive Optical Networks
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Classification of diffusion modes in single-particle tracking data: Feature-based versus deep-learning approach
• Quantitative Digital Microscopy with Deep Learning
• Deep learning-based super-resolution in coherent imaging systems
• Deep Learning for Feynman's Path Integral in Strong-Field Time-Dependent Dynamics
• Light scattering control in transmission and reflection with neural networks
• Self-supervised learning of hologram reconstruction using physics consistency
• Single-shot autofocusing of microscopy images using deep learning
• Deep learning in biomedical optics
• BGnet: Accurate and rapid background estimation in single-molecule localization microscopy with deep neural nets
• Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement
• Net2Vis -- A Visual Grammar for Automatically Generating Publication-Tailored CNN Architecture Visualizations
• cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM
• SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors
• Learned SPARCOM: Unfolded Deep Super-Resolution Microscopy
• Deep Iterative Reconstruction for Phase Retrieval
• Accurate phase retrieval of complex point spread functions with deep residual neural networks
• Machine learning for faster and smarter fluorescence lifetime imaging microscopy
• Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
• Recurrent neural network-based volumetric fluorescence microscopy
• Hybrid Machine Learning for Scanning Near-field Optical Spectroscopy
• On the interplay between physical and content priors in deep learning for computational imaging
• Far-field subwavelength acoustic imaging by deep learning
• Soft Metamaterials: Adaptation and Intelligence
• Deep learning-based photoacoustic imaging of vascular network through thick porous media
• Ghost imaging based on Y-net: a dynamic coding and conjugate-decoding approach
• Spatial images from temporal data
• Analysis of Super-resolution Single Molecule Localization Microscopy Data: a tutorial
• Deep learning-enabled image quality control in tomographic reconstruction: Robust optical diffraction tomography
• Harnessing Artificial Intelligence To Reduce Phototoxicity in Live Imaging
• Single-Pixel Pattern Recognition with Coherent Nonlinear Optics
• Optimal Transport driven CycleGAN for Unsupervised Learning in Inverse Problems
• Imaging arbitrary incoherent source distributions with near quantum-limited resolution
• Deep Learning for Biomedical Image Reconstruction: A Survey
• Virtual organelle self-coding for fluorescence imaging via adversarial learning
• Toward a Thinking Microscope: Deep Learning in Optical Microscopy and Image Reconstruction
• Two-particle sub-wavelength Quantum Correlation Microscopy
• Deep learning-based super-resolution fluorescence microscopy on small datasets
• Deep Learning Techniques for Inverse Problems in Imaging
• High-throughput, high-resolution registration-free generated adversarial network microscopy
• A New Deep Learning Method for Image Deblurring in Optical Microscopic Systems
• 3D Deep Learning Enables Fast Imaging of Spines through Scattering Media by Temporal Focusing Microscopy
• Learning an optimal PSF-pair for ultra-dense 3D localization microscopy
• Memory in quantum dot blinking
• Artefact removal in ground truth and noise model deficient sub-cellular nanoscopy images using auto-encoder deep learning
• Dipole-Spread Function Engineering for 6D Super-Resolution Microscopy
• Teaching deep neural networks to localize single molecules for super-resolution microscopy
• LensPlus: A High Space-bandwidth Optical Imaging Technique
• Single frame wide-field Nanoscopy based on Ghost Imaging via Sparsity Constraints (GISC Nanoscopy)
• Single-shot structured illumination microscopy
• Deep Phase Shifter for Quantitative Phase Imaging
• Weighted-CEL0 sparse regularisation for molecule localisation in super-resolution microscopy with Poisson data
• Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks
• Reconstructing Interpretable Features in Computational Super-Resolution microscopy via Regularized Latent Search
• From Stars to Molecules: AI Guided Device-Agnostic Super-Resolution Imaging
• Advancing biological super-resolution microscopy through deep learning: a brief review
• Sparse super resolution is Lipschitz continuous
• Fast and robust multiplane single molecule localization microscopy using deep neural network
• Deep Learning-Assisted Localisation of Nanoparticles in synthetically generated two-photon microscopy images
• Universal computational super-resolution framework for imaging of quantum dots
• Calibration-free single-frame super-resolution fluorescence microscopy
• Potential quality improvement of stochastic optical localization nanoscopy images obtained by frame by frame localization algorithms
• Augmented Equivariant Attention Networks for Microscopy Image Reconstruction
• Deep Unrolled Recovery in Sparse Biological Imaging