17 papers
U-Space: Uncovering When and Why Uncertainty Arises in Language Models
Tobias Braun, Nils Loose, Alexander Herzog +4
Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust…
The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models
Tobias Braun, Jonas Henry Grebe, Emil Sivic +4
Multimodal models are increasingly shifting toward unified architectures that understand and generate text, images, and other modalities within a shared conversational context. Thi…
Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating
Tobias Braun, Jonas Grebe, Louis Rethfeld +1
The widespread adoption of generative AI enables students to outsource cognitive effort to increasingly capable assistants, creating an illusion of competence while undermining the…
VETO: Towards Protecting Images From Frontier AI Editing
Jonas Grebe, Hossein Shakibania, Tobias Braun +2
The rise of powerful, accessible image-editing models such as FLUX.2 has brought high-fidelity editing within broad reach. Their capabilities now extend beyond localized modificati…
Obliviate: Erasing Concepts from Autoregressive Image Generation Models
Hossein Shakibania, Jonas Henry Grebe, Tobias Braun +4
The widespread adoption of generative AI models has intensified concerns about misuse, including the creation of unsafe or disturbing imagery. To mitigate such issues, several conc…
GEM: Geometric Erasure by Contrastive Velocity Matching in Rectified Flows
Jonas Henry Grebe, Tobias Braun, Anna Rohrbach +1
While the rapid adoption of multimodal generative models offers immense potential, it has also increased the risks of harmful content synthesis, deepfakes, and copyright infringeme…
Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models
Tobias Braun, Jonas Henry Grebe, Hossein Shakibania +2
Unified autoregressive models (UAMs) are transformer models that generate text as well as image tokens within a single autoregressive pass. Shared parameters and a multimodal vocab…
SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring
Hector G. Rodriguez, Marcus Rohrbach
Multimodal large language models (MLLMs) achieve ever-stronger performance on visual-language tasks. Even as traditional visual question answering (VQA) benchmarks approach saturat…
Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents
Nishad Singhi, Christian Bialas, Snehal Jauhri +4
Building generalist embodied agents capable of solving complex real-world tasks remains a fundamental challenge in AI. Multimodal Large Language Models (MLLMs) have significantly a…
HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models
Reihaneh Zohrabi, Hosein Hasani, Akshita Gupta +3
Large vision-language models can produce object hallucinations in image descriptions, highlighting the need for effective detection and mitigation strategies. Prior work commonly r…
VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking
Mark Rothermel, Marcus Kornmann, Marcus Rohrbach +1
The growing scale of online misinformation urgently demands Automated Fact-Checking (AFC). Existing benchmarks for evaluating AFC systems, however, are largely limited in terms of…
Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detection
Reihaneh Zohrabi, Hosein Hasani, Mahdieh Soleymani Baghshah +3
Out-of-distribution (OOD) detection is crucial for ensuring the reliability and safety of machine learning models in real-world applications, where they frequently face data distri…
Variational Visual Question Answering for Uncertainty-Aware Selective Prediction
Tobias Jan Wieczorek, Nathalie Daun, Mohammad Emtiyaz Khan +1
Despite remarkable progress in recent years, Vision Language Models (VLMs) remain prone to overconfidence and hallucinations on tasks such as Visual Question Answering (VQA) and Vi…
Erased but Not Forgotten: How Backdoors Compromise Concept Erasure
Tobias Braun, Jonas Henry Grebe, Marcus Rohrbach +2
The expansion of text-to-image diffusion models has raised concerns about harmful outputs, from fabricated depictions of public figures to sexually explicit imagery. To mitigate su…
When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning
Nishad Singhi, Hritik Bansal, Arian Hosseini +4
Scaling test-time compute has emerged as a key strategy for enhancing the reasoning capabilities of large language models (LLMs), particularly in tasks like mathematical problem-so…
VDial: Unification of Video and Visual Dialog via Multimodal Experts
Adnen Abdessaied, Anna Rohrbach, Marcus Rohrbach +1
We present VDial - a novel expert-based model specifically geared towards simultaneously handling image and video input data for multimodal conversational tasks. Current multim…
DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts
Tobias Braun, Mark Rothermel, Marcus Rohrbach +1
The proliferation of disinformation demands reliable and scalable fact-checking solutions. We present Dynamic Evidence-based FAct-checking with Multimodal Experts (DEFAME), a modul…