VLMs Can Aggregate Scattered Training Patches

Zhanhui Zhou, Lingjie Chen, Chao Yang +1

One way to mitigate risks in vision-language models (VLMs) is to remove dangerous samples in their training data. However, such data moderation can be easily bypassed when harmful…