Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learning

Abstract

Convolutional neural networks (CNN) have achieved state of the art performance on both classification and segmentation tasks. Applying CNNs to microscopy images is challenging due to the lack of datasets labeled at the single cell level. We extend the application of CNNs to microscopy image classification and segmentation using multiple instance learning (MIL). We present the adaptive Noisy-AND MIL pooling function, a new MIL operator that is robust to outliers. Combining CNNs with MIL enables training CNNs using full resolution microscopy images with global labels. We base our approach on the similarity between the aggregation function used in MIL and pooling layers used in CNNs. We show that training MIL CNNs end-to-end outperforms several previous methods on both mammalian and yeast microscopy images without requiring any segmentation steps.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Fully Convolutional Networks for Semantic Segmentation
• Striving for Simplicity: The All Convolutional Net
• Going Deeper with Convolutions
• Fully Convolutional Multi-Class Multiple Instance Learning

Cited by in corpus (37)

• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Applications of Deep Learning and Reinforcement Learning to Biological Data
• Attention-based Deep Multiple Instance Learning
• Micro-Net: A unified model for segmentation of various objects in microscopy images
• RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification
• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models
• Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions
• Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis
• KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images
• A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
• Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation
• Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classification
• Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology
• Multi-scale Cloud Detection in Remote Sensing Images using a Dual Convolutional Neural Network
• Learning image representations for anomaly detection: application to discovery of histological alterations in drug development
• An embarrassingly simple approach to neural multiple instance classification
• Thyroid Cancer Malignancy Prediction From Whole Slide Cytopathology Images
• Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification
• Hard Sample Mining for the Improved Retraining of Automatic Speech Recognition
• Learn to segment single cells with deep distance estimator and deep cell detector
• Using Multiple Instance Learning to Build Multimodal Representations
• Deep Instance-Level Hard Negative Mining Model for Histopathology Images
• Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology
• Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection
• pyLEMMINGS: Large Margin Multiple Instance Classification and Ranking for Bioinformatics Applications
• Computer-Assisted Analysis of Biomedical Images
• Multi-attention Networks for Temporal Localization of Video-level Labels
• Multiple Instance Hybrid Estimator for Hyperspectral Target Characterization and Sub-pixel Target Detection
• Bi-capacity Choquet Integral for Sensor Fusion with Label Uncertainty
• Learning from Web Data with Self-Organizing Memory Module
• Vessel-CAPTCHA: an efficient learning framework for vessel annotation and segmentation
• COVID-view: Diagnosis of COVID-19 using Chest CT
• User Personalized Satisfaction Prediction via Multiple Instance Deep Learning
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images
• Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks
• mil-benchmarks: Standardized Evaluation of Deep Multiple-Instance Learning Techniques