Attention-based Deep Multiple Instance Learning

Abstract

Multiple instance learning (MIL) is a variation of supervised learning where a single class label is assigned to a bag of instances. In this paper, we state the MIL problem as learning the Bernoulli distribution of the bag label where the bag label probability is fully parameterized by neural networks. Furthermore, we propose a neural network-based permutation-invariant aggregation operator that corresponds to the attention mechanism. Notably, an application of the proposed attention-based operator provides insight into the contribution of each instance to the bag label. We show empirically that our approach achieves comparable performance to the best MIL methods on benchmark MIL datasets and it outperforms other methods on a MNIST-based MIL dataset and two real-life histopathology datasets without sacrificing interpretability.

arXiv commentICML 2018 paper, code source: https://github.com/AMLab-Amsterdam/AttentionDeepMIL

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• A Survey on Deep Learning in Medical Image Analysis
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• A Structured Self-attentive Sentence Embedding
• Language Modeling with Gated Convolutional Networks
• Multiple Instance Learning: A Survey of Problem Characteristics and Applications
• Revisiting Multiple Instance Neural Networks
• Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learning
• Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
• Multiple Instance Learning with Bag Dissimilarities
• Label Stability in Multiple Instance Learning

Cited by in corpus (92)

• Deep neural network models for computational histopathology: A survey
• Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks
• Deep Learning for Whole Slide Image Analysis: An Overview
• Monte-Carlo Sampling applied to Multiple Instance Learning for Histological Image Classification
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• Neural Image Compression for Gigapixel Histopathology Image Analysis
• RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification
• Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
• DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer
• Streaming convolutional neural networks for end-to-end learning with multi-megapixel images
• Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions
• Weakly Labelled AudioSet Tagging with Attention Neural Networks
• Detecting Parkinsonian Tremor from IMU Data Collected In-The-Wild using Deep Multiple-Instance Learning
• Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis
• Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides
• MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data
• Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists
• Multiple instance learning with graph neural networks
• HEROHE Challenge: assessing HER2 status in breast cancer without immunohistochemistry or in situ hybridization
• All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification
• WeGleNet: A Weakly-Supervised Convolutional Neural Network for the Semantic Segmentation of Gleason Grades in Prostate Histology Images
• Sequential Weakly Labeled Multi-Activity Localization and Recognition on Wearable Sensors using Recurrent Attention Networks
• Multi-Modal Multi-Instance Learning for Retinal Disease Recognition
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Self-learning for weakly supervised Gleason grading of local patterns
• An attention-based multi-resolution model for prostate whole slide imageclassification and localization
• Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
• Weakly Supervised Attention-based Models Using Activation Maps for Citrus Mite and Insect Pest Classification
• Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning
• COVID-MobileXpert: On-Device COVID-19 Patient Triage and Follow-up using Chest X-rays
• PS-DeVCEM: Pathology-sensitive deep learning model for video capsule endoscopy based on weakly labeled data
• Sharing Pain: Using Pain Domain Transfer for Video Recognition of Low Grade Orthopedic Pain in Horses
• Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images
• Medical Time Series Classification with Hierarchical Attention-based Temporal Convolutional Networks: A Case Study of Myotonic Dystrophy Diagnosis
• An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization
• Robust partial Fourier reconstruction for diffusion-weighted imaging using a recurrent convolutional neural network
• Set2Graph: Learning Graphs From Sets
• Machine Learning for Cutting Planes in Integer Programming: A Survey
• An embarrassingly simple approach to neural multiple instance classification
• Thyroid Cancer Malignancy Prediction From Whole Slide Cytopathology Images
• Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images
• Exchangeable deep neural networks for set-to-set matching and learning
• Synergistic Learning of Lung Lobe Segmentation and Hierarchical Multi-Instance Classification for Automated Severity Assessment of COVID-19 in CT Images
• Histopathological Image Classification based on Self-Supervised Vision Transformer and Weak Labels
• Attention-based Multi-instance Neural Network for Medical Diagnosis from Incomplete and Low Quality Data
• Self-Attentive Document Interaction Networks for Permutation Equivariant Ranking
• Min-max Entropy for Weakly Supervised Pointwise Localization
• Psychophysiological Arousal in Young Children Who Stutter: An Interpretable AI Approach
• Needles in Haystacks: On Classifying Tiny Objects in Large Images
• Data and Knowledge Co-driving for Cancer Subtype Classification on Multi-Scale Histopathological Slides
• Address Instance-level Label Prediction in Multiple Instance Learning
• DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction
• Leveraging Unlabelled Data in Multiple-Instance Learning Problems for Improved Detection of Parkinsonian Tremor in Free-Living Conditions
• Pi-PE: A Pipeline for Pulmonary Embolism Detection using Sparsely Annotated 3D CT Images
• A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning
• Orientation-Disentangled Unsupervised Representation Learning for Computational Pathology
• MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection
• On the Relevance of Temporal Features for Medical Ultrasound Video Recognition
• Unsupervised deep learning for grading of age-related macular degeneration using retinal fundus images
• Globally-Aware Multiple Instance Classifier for Breast Cancer Screening
• Dual-stream Maximum Self-attention Multi-instance Learning
• Multi-Instance Learning by Utilizing Structural Relationship among Instances
• Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms
• Deformable Gabor Feature Networks for Biomedical Image Classification
• Learning with Sets in Multiple Instance Regression Applied to Remote Sensing
• Is it feasible to detect FLOSS version release events from textual messages? A case study on Stack Overflow
• AMI-Net+: A Novel Multi-Instance Neural Network for Medical Diagnosis from Incomplete and Imbalanced Data
• An Attention-based Weakly Supervised framework for Spitzoid Melanocytic Lesion Diagnosis in WSI
• Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection
• Detecting cutaneous basal cell carcinomas in ultra-high resolution and weakly labelled histopathological images
• Challenging mitosis detection algorithms: Global labels allow centroid localization
• BERT for Large-scale Video Segment Classification with Test-time Augmentation
• Foveation for Segmentation of Ultra-High Resolution Images
• Generative-Discriminative Complementary Learning
• Spotting insects from satellites: modeling the presence of Culicoides imicola through Deep CNNs
• Learning Maximally Predictive Prototypes in Multiple Instance Learning
• Kernel Self-Attention in Deep Multiple Instance Learning
• Image Captioning with Compositional Neural Module Networks
• Improving Interpretability for Computer-aided Diagnosis tools on Whole Slide Imaging with Multiple Instance Learning and Gradient-based Explanations
• Cross-scale Attention Model for Acoustic Event Classification
• Multi-attention Networks for Temporal Localization of Video-level Labels
• Improving Robustness of Malware Classifiers using Adversarial Strings Generated from Perturbed Latent Representations
• Multiple Instance Learning for Uplift Modeling
• META${}^2$: Memory-efficient taxonomic classification and abundance estimation for metagenomics with deep learning
• In Defense of LSTMs for Addressing Multiple Instance Learning Problems
• Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks
• Deep DA for Ordinal Regression of Pain Intensity Estimation Using Weakly-Labeled Videos
• Reliable Liver Fibrosis Assessment from Ultrasound using Global Hetero-Image Fusion and View-Specific Parameterization
• Self-Guided Multiple Instance Learning for Weakly Supervised Disease Classification and Localization in Chest Radiographs
• Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data
• Deep Learning Derived Histopathology Image Score for Increasing Phase 3 Clinical Trial Probability of Success
• On the detection of Out-Of-Distribution samples in Multiple Instance Learning