6 papers
Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents
Suman Navaratnarajah, Taehyoung Kim, Jona Ruthardt +5
Multimodal Large Language Models (MLLMs) are emerging as core reasoning modules for embodied agents, yet it remains unclear how well general-purpose models can solve long-horizon e…
Steerable Visual Representations
Jona Ruthardt, Manu Gaur, Deva Ramanan +2
Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tasks such as retrieval, classification,…
Do Instance Priors Help Weakly Supervised Semantic Segmentation?
Anurag Das, Anna Kukleva, Xinting Hu +2
Semantic segmentation requires dense pixel-level annotations, which are costly and time-consuming to acquire. To address this, we present SeSAM, a framework that uses a foundationa…
TULIP: Token-length Upgraded CLIP
Ivona Najdenkoska, Mohammad Mahdi Derakhshani, Yuki M. Asano +3
We address the challenge of representing long captions in vision-language models, such as CLIP. By design these models are limited by fixed, absolute positional encodings, restrict…
TWIST & SCOUT: Grounding Multimodal LLM-Experts by Forget-Free Tuning
Aritra Bhowmik, Mohammad Mahdi Derakhshani, Dennis Koelma +3
Spatial awareness is key to enable embodied multimodal AI systems. Yet, without vast amounts of spatial supervision, current Multimodal Large Language Models (MLLMs) struggle at th…
Auto-Vocabulary Semantic Segmentation
Osman Ãlger, Maksymilian Kulicki, Yuki Asano +1
Open-Vocabulary Segmentation (OVS) methods are capable of performing semantic segmentation without relying on a fixed vocabulary, and in some cases, without training or fine-tuning…