Controlling for Stereotypes in Multimodal Language Model Evaluation

Manuj Malik, Richard Johansson

We propose a methodology and design two benchmark sets for measuring to what extent language-and-vision language models use the visual signal in the presence or absence of stereoty…