4 papers
Mind the Heads: Topological Representation Alignment for Multimodal LLMs
Davide Caffagni, Alberto Compagnoni, Federico Melis +5
Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal representations toward those of an…
Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models
Davide Caffagni, Sara Sarto, Marcella Cornia +5
Multimodal Large Language Models (MLLMs) have recently demonstrated impressive capabilities in connecting vision and language, yet their proficiency in fundamental visual reasoning…
Lost in Translation? Vocabulary Alignment for Source-Free Adaptation in Open-Vocabulary Semantic Segmentation
Silvio Mazzucco, Carl Persson, Mattia Segu +4
We introduce VocAlign, a novel source-free domain adaptation framework specifically designed for VLMs in open-vocabulary semantic segmentation. Our method adopts a student-teacher…
Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation
Reza Qorbani, Gianluca Villani, Theodoros Panagiotakopoulos +8
Open-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries, providing versatile performance on no…