Interpreting and Controlling Vision Foundation Models via Text Explanations

Haozhe Chen, Junfeng Yang, Carl Vondrick +1

Large-scale pre-trained vision foundation models, such as CLIP, have become de facto backbones for various vision tasks. However, due to their black-box nature, understanding the u…