Similarity and Quality Metrics for MR Image-To-Image Translation

Abstract

Image-to-image translation can create large impact in medical imaging, as images can be synthetically transformed to other modalities, sequence types, higher resolutions or lower noise levels. To ensure patient safety, these methods should be validated by human readers, which requires a considerable amount of time and costs. Quantitative metrics can effectively complement such studies and provide reproducible and objective assessment of synthetic images. If a reference is available, the similarity of MR images is frequently evaluated by SSIM and PSNR metrics, even though these metrics are not or too sensitive regarding specific distortions. When reference images to compare with are not available, non-reference quality metrics can reliably detect specific distortions, such as blurriness. To provide an overview on distortion sensitivity, we quantitatively analyze 11 similarity (reference) and 12 quality (non-reference) metrics for assessing synthetic images. We additionally include a metric on a downstream segmentation task. We investigate the sensitivity regarding 11 kinds of distortions and typical MR artifacts, and analyze the influence of different normalization methods on each metric and distortion. Finally, we derive recommendations for effective usage of the analyzed similarity and quality metrics for evaluation of image-to-image translation models.

44 pages (main: 22 pages, 3 figures, supplement: 22 pages, 15 figures)

References in corpus (8)

• scikit-image: Image processing in Python
• Metrics reloaded: Recommendations for image analysis validation
• Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
• Diffusion Probabilistic Models beat GANs on Medical Images
• Understanding metric-related pitfalls in image analysis validation
• One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputation
• Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study
• Five Pitfalls When Assessing Synthetic Medical Images with Reference Metrics