StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment

Younghyun Kim, Jongheon Jeong, Sangkyung Kwak +3

Learning robust representations from data often requires scale, which has led to the success of recent zero-shot models such as CLIP. However, the obtained robustness can easily be…