12 papers
Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models
Niclas Lietzow, Danielle Bitterman, Carsten Eickhoff +2
Vision-language models must reconcile visual evidence with memorized world knowledge when the two conflict. How they resolve this conflict shapes the reliability of multimodal syst…
UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languages
Tassallah Abdullahi, Macton Mgonzo, Mardiyyah Oduwole +4
Current guardian models are predominantly Western-centric and optimized for high-resource languages, leaving low-resource African languages vulnerable to evolving harms, cross-ling…
PubMed-Ophtha: An open resource for training ophthalmology vision-language models on scientific literature
Verena Jasmin Hallitschke, Carsten Eickhoff, Philipp Berens
Vision-language models hold considerable promise for ophthalmology, but their development depends on large-scale, high-quality image-text datasets that remain scarce. We present Pu…
Mechanisms of Prompt-Induced Hallucination in Vision-Language Models
William Rudman, Michal Golovanevsky, Dana Arad +4
Large vision-language models (VLMs) are highly capable, yet often hallucinate by favoring textual prompts over visual evidence. We study this failure mode in a controlled object-co…
Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Models
Oliver McLaughlin, Daniel Shubin, Carsten Eickhoff +3
Vision-language models (VLMs) are increasingly adapted through domain-specific fine-tuning, yet it remains unclear whether this improves reasoning beyond superficial visual cues, p…
APP: Accelerated Path Patching with Task-Specific Pruning
Frauke Andersen, William Rudman, Ruochen Zhang +1
Circuit discovery is a key step in many mechanistic interpretability pipelines. Current methods, such as Path Patching, are computationally expensive and have limited in-depth circ…