Harnessing Textual Refusal Directions for Multimodal Safety

Moreno D'Incà, Nicu Sebe, Massimiliano Mancini

Safety in Multimodal Large Language Models (MLLMs) depends on unsafe multimodal data for post-training alignment or activation steering. However, such data is hard to collect, must…