TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning

Abstract

Processing of medical images such as MRI or CT presents unique challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment of volumes. We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. TorchIO transforms can be composed, reproduced, traced and extended. We provide multiple generic preprocessing and augmentation operations as well as simulation of MRI-specific artifacts. Source code, comprehensive tutorials and extensive documentation for TorchIO can be found at https://torchio.rtfd.io/. The package can be installed from the Python Package Index running 'pip install torchio'. It includes a command-line interface which allows users to apply transforms to image files without using Python. Additionally, we provide a graphical interface within a TorchIO extension in 3D Slicer to visualize the effects of transforms. TorchIO was developed to help researchers standardize medical image processing pipelines and allow them to focus on the deep learning experiments. It encourages open science, as it supports reproducibility and is version controlled so that the software can be cited precisely. Due to its modularity, the library is compatible with other frameworks for deep learning with medical images.

Published in Computer Methods and Programs in Biomedicine

References in corpus (16)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• The NumPy array: a structure for efficient numerical computation
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Direct Estimation of Spinal Cobb Angles by Structured Multi-Output Regression
• DLTK: State of the Art Reference Implementations for Deep Learning on Medical Images
• A Learning Strategy for Contrast-agnostic MRI Segmentation
• pymia: A Python package for data handling and evaluation in deep learning-based medical image analysis
• Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images
• Unsupervised End-to-end Learning for Deformable Medical Image Registration
• MONAIfbs: MONAI-based fetal brain MRI deep learning segmentation
• Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient
• Simulation of Brain Resection for Cavity Segmentation Using Self-Supervised and Semi-Supervised Learning
• Partial Volume Segmentation of Brain MRI Scans of any Resolution and Contrast
• Eisen: a python package for solid deep learning

Cited by in corpus (55)

• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• Federated Learning Enables Big Data for Rare Cancer Boundary Detection
• Classification of Brain Tumours in MR Images using Deep Spatiospatial Models
• PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation
• medigan: a Python library of pretrained generative models for medical image synthesis
• pymia: A Python package for data handling and evaluation in deep learning-based medical image analysis
• GaNDLF: A Generally Nuanced Deep Learning Framework for Scalable End-to-End Clinical Workflows in Medical Imaging
• Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets
• A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections
• Scribble-based Domain Adaptation via Co-segmentation
• Earth System Data Cubes: Avenues for advancing Earth system research
• Artifact Reduction in 3D and 4D Cone-beam Computed Tomography Images with Deep Learning -- A Review
• Complex Network for Complex Problems: A comparative study of CNN and Complex-valued CNN
• Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation
• DS6, Deformation-aware Semi-supervised Learning: Application to Small Vessel Segmentation with Noisy Training Data
• FastSurfer-HypVINN: Automated sub-segmentation of the hypothalamus and adjacent structures on high-resolutional brain MRI
• Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning models
• Guided Reconstruction with Conditioned Diffusion Models for Unsupervised Anomaly Detection in Brain MRIs
• Fibroglandular Tissue Segmentation in Breast MRI using Vision Transformers -- A multi-institutional evaluation
• Simulation of Brain Resection for Cavity Segmentation Using Self-Supervised and Semi-Supervised Learning
• Acute ischemic stroke lesion segmentation in non-contrast CT images using 3D convolutional neural networks
• Axial multi-layer perceptron architecture for automatic segmentation of choroid plexus in multiple sclerosis
• MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis
• Exploring contrast generalisation in deep learning-based brain MRI-to-CT synthesis
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Automated Olfactory Bulb Segmentation on High Resolutional T2-Weighted MRI
• Model-based inexact graph matching on top of CNNs for semantic scene understanding
• TransMed: Transformers Advance Multi-modal Medical Image Classification
• Retrospective Motion Correction of MR Images using Prior-Assisted Deep Learning
• Shape-aware Segmentation of the Placenta in BOLD Fetal MRI Time Series
• ResAtom System: Protein and Ligand Affinity Prediction Model Based on Deep Learning
• Extreme Cardiac MRI Analysis under Respiratory Motion: Results of the CMRxMotion Challenge
• Systematic Review and Meta-analysis of AI-driven MRI Motion Artifact Detection and Correction
• Vision-Language Model-Based Semantic-Guided Imaging Biomarker for Lung Nodule Malignancy Prediction
• RFR-WWANet: Weighted Window Attention-Based Recovery Feature Resolution Network for Unsupervised Image Registration
• Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge
• UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation
• Automated MRI Quality Assessment of Brain T1-weighted MRI in Clinical Data Warehouses: A Transfer Learning Approach Relying on Artefact Simulation
• Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets
• Deep Learning Enables Reduced Gadolinium Dose for Contrast-Enhanced Blood-Brain Barrier Opening
• Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation
• Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data
• Eisen: a python package for solid deep learning
• The role of MRI physics in brain segmentation CNNs: achieving acquisition invariance and instructive uncertainties
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning
• PULASki: Learning inter-rater variability using statistical distances to improve probabilistic segmentation
• Team NeuroPoly: Description of the Pipelines for the MICCAI 2021 MS New Lesions Segmentation Challenge
• ATOMMIC: An Advanced Toolbox for Multitask Medical Imaging Consistency to facilitate Artificial Intelligence applications from acquisition to analysis in Magnetic Resonance Imaging
• Improving Quality Control Of MRI Images Using Synthetic Motion Data
• 3D-OOCS: Learning Prostate Segmentation with Inductive Bias
• 2D Multi-Class Model for Gray and White Matter Segmentation of the Cervical Spinal Cord at 7T
• Interpretable Prediction of Lymph Node Metastasis in Rectal Cancer MRI Using Variational Autoencoders
• PyKale: Knowledge-Aware Machine Learning from Multiple Sources in Python
• Segmenting infant brains across magnetic fields: Domain randomization and annotation curation in ultra-low field MRI
• Quantitative mapping from conventional MRI using self-supervised physics-guided deep learning: applications to a large-scale, clinically heterogeneous dataset