6 papers
SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2
Reinforcement learning (RL) with verifiable rewards (RLVR) has demonstrated the great potential of enhancing the reasoning abilities in multimodal large language models (MLLMs). Ho…
R-CoV: Region-Aware Chain-of-Verification for Alleviating Object Hallucinations in LVLMs
Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2
Large vision-language models (LVLMs) have demonstrated impressive performance in various multimodal understanding and reasoning tasks. However, they still struggle with object hall…
Gatekeeper: Improving Model Cascades Through Confidence Tuning
Stephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha +5
Large-scale machine learning models deliver strong performance across a wide range of tasks but come with significant computational and resource constraints. To mitigate these chal…
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
Shangbin Feng, Zifeng Wang, Yike Wang +9
We propose Model Swarms, a collaborative search algorithm to adapt LLMs via swarm intelligence, the collective behavior guiding individual systems. Specifically, Model Swarms start…
Test-Time Visual In-Context Tuning
Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2
Visual in-context learning (VICL), as a new paradigm in computer vision, allows the model to rapidly adapt to various tasks with only a handful of prompts and examples. While effec…
Extracting Training Data from Document-Based VQA Models
Francesco Pinto, Nathalie Rauschmayr, Florian Tramèr +2
Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provide…