Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets

Abstract

Determining whether two sets of images belong to the same or different distributions or domains is a crucial task in modern medical image analysis and deep learning; for example, to evaluate the output quality of image generative models. Currently, metrics used for this task either rely on the (potentially biased) choice of some downstream task, such as segmentation, or adopt task-independent perceptual metrics (e.g., Fréchet Inception Distance/FID) from natural imaging, which we show insufficiently capture anatomical features. To this end, we introduce a new perceptual metric tailored for medical images, FRD (Fréchet Radiomic Distance), which utilizes standardized, clinically meaningful, and interpretable image features. We show that FRD is superior to other image distribution metrics for a range of medical imaging applications, including out-of-domain (OOD) detection, the evaluation of image-to-image translation (by correlating more with downstream task performance as well as anatomical consistency and realism), and the evaluation of unconditional image generation. Moreover, FRD offers additional benefits such as stability and computational efficiency at low sample sizes, sensitivity to image corruptions and adversarial attacks, feature interpretability, and correlation with radiologist-perceived image quality. Additionally, we address key gaps in the literature by presenting an extensive framework for the multifaceted evaluation of image similarity metrics in medical imaging -- including the first large-scale comparative study of generative models for medical image translation -- and release an accessible codebase to facilitate future research. Our results are supported by thorough experiments spanning a variety of datasets, modalities, and downstream tasks, highlighting the broad potential of FRD for medical image analysis.

Codebase for FRD computation: https://github.com/RichardObi/frd-score. Codebase for medical image similarity metric evaluation framework: https://github.com/mazurowski-lab/medical-image-similarity-metrics

References in corpus (11)

• Domain Adaptation for Medical Image Analysis: A Survey
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• Metrics reloaded: Recommendations for image analysis validation
• Understanding metric-related pitfalls in image analysis validation
• Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN
• Foundation Models in Radiology: What, How, When, Why and Why Not
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• A large-scale multicenter breast cancer DCE-MRI benchmark dataset with expert segmentations
• medigan: a Python library of pretrained generative models for medical image synthesis
• High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection
• Deep Learning for Breast MRI Style Transfer with Limited Training Data