4 papers
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
DoYoung Kim, Changkyu Choi, Adín Ramírez Rivera +2
Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inhere…
Suppressing Non-Semantic Noise in Masked Image Modeling Representations
Martine Hjelkrem-Tan, Marius Aasan, Rwiddhi Chakraborty +3
Masked Image Modeling (MIM) has become a ubiquitous self-supervised vision paradigm. In this work, we show that MIM objectives cause the learned representations to retain non-seman…
Differentiable Hierarchical Visual Tokenization
Marius Aasan, Martine Hjelkrem-Tan, Nico Catalano +2
Vision Transformers rely on fixed patch tokens that ignore the spatial and semantic structure of images. In this work, we introduce an end-to-end differentiable tokenizer that adap…
DocVXQA: Context-Aware Visual Explanations for Document Question Answering
Mohamed Ali Souibgui, Changkyu Choi, Andrey Barsky +3
We propose DocVXQA, a novel framework for visually self-explainable document question answering. The framework is designed not only to produce accurate answers to questions but als…