Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts

Nimrod Shabtay, Asaf Yehudai, Avihu Dekel +2

Vision-language models (VLMs) struggle with in context learning compared to their textual counterparts. Our preliminary study shows that VLMs rely on textual shortcuts and underuti…