Lost in Contrast: Instruction-Preserving Decoding for Reliable Visual Question Answering

Haonan Zhang, Ting Jin, Wenzhong Tang +1

Visual contrastive decoding corrects answers using the prediction difference between the original image and a reference condition, yet the same contrast also turns supported absten…