Comparison of fine-tuning strategies for transfer learning in medical image classification

Abstract

In the context of medical imaging and machine learning, one of the most pressing challenges is the effective adaptation of pre-trained models to specialized medical contexts. Despite the availability of advanced pre-trained models, their direct application to the highly specialized and diverse field of medical imaging often falls short due to the unique characteristics of medical data. This study provides a comprehensive analysis on the performance of various fine-tuning methods applied to pre-trained models across a spectrum of medical imaging domains, including X-ray, MRI, Histology, Dermoscopy, and Endoscopic surgery. We evaluated eight fine-tuning strategies, including standard techniques such as fine-tuning all layers or fine-tuning only the classifier layers, alongside methods such as gradually unfreezing layers, regularization based fine-tuning and adaptive learning rates. We selected three well-established CNN architectures (ResNet-50, DenseNet-121, and VGG-19) to cover a range of learning and feature extraction scenarios. Although our results indicate that the efficacy of these fine-tuning methods significantly varies depending on both the architecture and the medical imaging type, strategies such as combining Linear Probing with Full Fine-tuning resulted in notable improvements in over 50% of the evaluated cases, demonstrating general effectiveness across medical domains. Moreover, Auto-RGN, which dynamically adjusts learning rates, led to performance enhancements of up to 11% for specific modalities. Additionally, the DenseNet architecture showed more pronounced benefits from alternative fine-tuning approaches compared to traditional full fine-tuning. This work not only provides valuable insights for optimizing pre-trained models in medical image analysis but also suggests the potential for future research into more advanced architectures and fine-tuning methods.

Accepted at Image and Vision Computing

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Covid-19: Automatic detection from X-Ray images utilizing Transfer Learning with Convolutional Neural Networks
• SurfaceNet: Adversarial SVBRDF Estimation from a Single Image
• Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
• Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
• Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
• Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization

Cited by in corpus (5)

• Voice control interface for surgical robot assistants
• Adaptive transfer learning for surgical tool presence detection in laparoscopic videos through gradual freezing fine-tuning
• Bio-inspired fine-tuning for selective transfer learning in image classification
• A hierarchical framework for collision avoidance in robot-assisted minimally invasive surgery
• Embedded Image-to-Image Translation for Efficient Sim-to-Real Transfer in Learning-based Robot-Assisted Soft Manipulation