5 papers
Causal Attribution via Activation Patching
Amirmohammad Izadi, Mohammadali Banayeeanzade, Alireza Mirrokni +4
Attribution methods for Vision Transformers (ViTs) aim to identify image regions that influence model predictions, but producing faithful and well-localized attributions remains ch…
Eye-Q: A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning
Ali Najar, Alireza Mirrokni, Arshia Izadyari +5
Vision-Language Models (VLMs) have achieved strong performance on standard vision-language benchmarks, yet often rely on surface-level recognition rather than deeper reasoning. We…
Mechanistic Interpretability of Large-Scale Counting in LLMs through a System-2 Strategy
Hosein Hasani, Mohammadali Banayeeanzade, Ali Nafisi +5
Large language models (LLMs), despite strong performance on complex mathematical problems, exhibit systematic limitations in counting tasks. This issue arises from the architectura…
Understanding Counting Mechanisms in Large Language and Vision-Language Models
Hosein Hasani, Amirmohammad Izadi, Fatemeh Askari +4
Counting is one of the fundamental abilities of large language models (LLMs) and large vision-language models (LVLMs). This paper examines how these foundation models represent and…
Uncovering Grounding IDs: How External Cues Shape Multi-Modal Binding
Hosein Hasani, Amirmohammad Izadi, Fatemeh Askari +4
Large vision-language models (LVLMs) perform well on multimodal tasks, but their ability to reason and precisely align visual and textual information still has room for improvement…