From the 1 of 7 linked papers with an AI index.
7 papers
DiMaS: Distribution Matching for Steering Vision-Language-Action Models
Pegah Khayatan, Sara Meziane, Jayneel Parekh +1
The paper introduces DiMaS, a distribution‑matching steering technique that adjusts the internal representations of flow‑matching vision‑language‑action models to achieve fine‑grai…
When Prompts Override Vision: Prompt-Induced Hallucinations in LVLMs
Pegah Khayatan, Jayneel Parekh, Arnaud Dapogny +3
Despite impressive progress in capabilities of large vision-language models (LVLMs), these systems remain vulnerable to hallucinations, i.e., outputs that are not grounded in the v…
Learning to Steer: Input-dependent Steering for Multimodal LLMs
Jayneel Parekh, Pegah Khayatan, Mustafa Shukor +3
Steering has emerged as a practical approach to enable post-hoc guidance of LLMs towards enforcing a specific behavior. However, it remains largely underexplored for multimodal LLM…
Analyzing Finetuning Representation Shift for Multimodal LLMs Steering
Pegah Khayatan, Mustafa Shukor, Jayneel Parekh +2
Multimodal LLMs (MLLMs) have reached remarkable levels of proficiency in understanding multimodal inputs. However, understanding and interpreting the behavior of such complex model…
One Wave To Explain Them All: A Unifying Perspective On Feature Attribution
Gabriel Kasmi, Amandine Brunetto, Thomas Fel +1
Feature attribution methods aim to improve the transparency of deep neural networks by identifying the input features that influence a model's decision. Pixel-based heatmaps have b…
Restyling Unsupervised Concept Based Interpretable Networks with Generative Models
Jayneel Parekh, Quentin Bouniot, Pavlo Mozharovskyi +2
Developing inherently interpretable models for prediction has gained prominence in recent years. A subclass of these models, wherein the interpretable network relies on learning hi…