9 papers
Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility
Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1
Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study how…
GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods
Sujay Belsare, Sudarshan Nikhil, Sushant Kumar +2
With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of expl…
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
Ding Zhang, Runtao Zhou, Wenqing Zheng +3
Graph Language Models (GLMs) have become a promising direction for adapting Large Language Models (LLMs) to graph learning tasks. By transforming graph topology and node informatio…
Do Vision Language Models Need to Process Image Tokens?
Sambit Ghosh, R. Venkatesh Babu, Chirag Agarwal
Vision Language Models (VLMs) have achieved remarkable success by integrating visual encoders with large language models (LLMs). While VLMs process dense image tokens across deep t…
SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future
Timothy B. Higgins, Antonios Mamalakis, Chirag Agarwal
Recent advances in visual-language models (VLMs) have led to significant improvements in a plethora of complex multimodal tasks like image captioning, report generation, and visual…
Polarity-Aware Probing for Quantifying Latent Alignment in Language Models
Sabrina Sadiekh, Elena Ericheva, Chirag Agarwal
Advances in unsupervised probes such as Contrast-Consistent Search (CCS), which reveal latent beliefs without relying on token outputs, raise the question of whether these methods…
EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding
Ashish Seth, Utkarsh Tyagi, Ramaneswaran Selvakumar +6
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in complex multimodal tasks. While MLLMs excel at visual perception and reasoning in third-person…
Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models
Ashish Seth, Dinesh Manocha, Chirag Agarwal
Large Vision-Language Models (LVLMs) have demonstrated remarkable performance in complex multimodal tasks. However, these models still suffer from hallucinations, particularly when…
Towards Operationalizing Right to Data Protection
Abhinav Java, Simra Shahid, Chirag Agarwal
The widespread practice of indiscriminate data scraping to fine-tune language models (LMs) raises significant legal and ethical concerns, particularly regarding compliance with dat…