Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-Image Encoders

Andrew Geng, Pin-Yu Chen

When evaluating the performance of a pre-trained model transferred to a downstream task, it is imperative to assess not only the in-distribution (ID) accuracy of the downstream mod…