Dr. No: Feature Steering for Abstention in Vision-Language Models

Farid Hasanov, Riccardo Renzulli, Bahram Zonooz +2

Large foundation models often answer even when the available visual or textual evidence is insufficient to support a reliable conclusion. This failure to abstain makes them hard to…