Improved Regularization and Robustness for Fine-tuning in Neural Networks

Dongyue Li, Hongyang R. Zhang

A widely used algorithm for transfer learning is fine-tuning, where a pre-trained model is fine-tuned on a target task with a small amount of labeled data. When the capacity of the…