VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classification

Abstract

Artificial Intelligence (AI) has the potential to revolutionize diagnosis and segmentation in medical imaging. However, development and clinical implementation face multiple challenges including limited data availability, lack of generalizability, and the necessity to incorporate multi-modal data effectively. A foundation model, which is a large-scale pre-trained AI model, offers a versatile base that can be adapted to a variety of specific tasks and contexts. Here, we present VIsualization and Segmentation Masked AutoEncoder (VIS-MAE), novel model weights specifically designed for medical imaging. Specifically, VIS-MAE is trained on a dataset of 2.5 million unlabeled images from various modalities (CT, MR, PET,X-rays, and ultrasound), using self-supervised learning techniques. It is then adapted to classification and segmentation tasks using explicit labels. VIS-MAE has high label efficiency, outperforming several benchmark models in both in-domain and out-of-domain applications. In addition, VIS-MAE has improved label efficiency as it can achieve similar performance to other models with a reduced amount of labeled training data (50% or 80%) compared to other pre-trained weights. VIS-MAE represents a significant advancement in medical imaging AI, offering a generalizable and robust solution for improving segmentation and classification tasks while reducing the data annotation workload. The source code of this work is available at https://github.com/lzl199704/VIS-MAE.

Accepted at MLMI@MICCAI (Workshop on Machine Learning in Medical Imaging at MICCAI 2024))

References in corpus (11)

• Automated Design of Deep Learning Methods for Biomedical Image Segmentation
• A Simple Framework for Contrastive Learning of Visual Representations
• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Segment Anything in Medical Images
• Confounding variables can degrade generalization performance of radiological deep learning models
• Recent advances and clinical applications of deep learning in medical image analysis
• Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm
• Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images
• SegVol: Universal and Interactive Volumetric Medical Image Segmentation
• A Systematic Collection of Medical Image Datasets for Deep Learning