Convolutional neural networks that teach microscopes how to image

Abstract

Deep learning algorithms offer a powerful means to automatically analyze the content of medical images. However, many biological samples of interest are primarily transparent to visible light and contain features that are difficult to resolve with a standard optical microscope. Here, we use a convolutional neural network (CNN) not only to classify images, but also to optimize the physical layout of the imaging device itself. We increase the classification accuracy of a microscope's recorded images by merging an optical model of image formation into the pipeline of a CNN. The resulting network simultaneously determines an ideal illumination arrangement to highlight important sample features during image acquisition, along with a set of convolutional weights to classify the detected images post-capture. We demonstrate our joint optimization technique with an experimental microscope configuration that automatically identifies malaria-infected cells with 5-10% higher accuracy than standard and alternative microscope lighting designs.

13 pages, 6 figures

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• A Survey on Deep Learning in Medical Image Analysis
• Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks
• Deep Convolutional Neural Networks for Microscopy-Based Point of Care Diagnostics
• ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements

Cited by in corpus (16)

• Low Photon Count Phase Retrieval Using Deep Learning
• Reliable deep-learning-based phase imaging with uncertainty quantification
• Artificial Neural Network with Physical Dynamic Metasurface Layer for Optimal Sensing
• J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction
• Intelligent Meta-Imagers: From Compressed to Learned Sensing
• Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopy
• PUERT: Probabilistic Under-sampling and Explicable Reconstruction Network for CS-MRI
• Flash Photography for Data-Driven Hidden Scene Recovery
• Imaging through glass diffusers using densely connected convolutional networks
• Quantitative Phase Imaging and Artificial Intelligence: A Review
• Optimal Physical Preprocessing for Example-Based Super-Resolution
• Increasing a microscope's effective field of view via overlapped imaging and machine learning
• Noise-Adaptive Intelligent Programmable Meta-Imager
• Deep Learning Techniques for Inverse Problems in Imaging
• Learning an optimal PSF-pair for ultra-dense 3D localization microscopy
• Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging