6 papers · 1 filter
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
Vatsal Gupta, Pranshu Pandya, Tushar Kataria +2
Language models, characterized by their black-box nature, often hallucinate and display sensitivity to input perturbations, causing concerns about trust. To enhance trust, it is im…
Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness
Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella +3
Chart question answering (CQA) is a crucial area of Visual Language Understanding. However, the robustness and consistency of current Visual Language Models (VLMs) in this field re…
MAPWise: Evaluating Vision-Language Models for Advanced Map Queries
Srija Mukhopadhyay, Abhishek Rajgaria, Prerana Khatiwada +2
Vision-language models (VLMs) excel at tasks requiring joint understanding of visual and linguistic information. A particularly promising yet under-explored application for these m…
Knowledge-Aware Reasoning over Multimodal Semi-structured Tables
Suyash Vardhan Mathur, Jainit Sushil Bafna, Kunal Kartik +5
Existing datasets for tabular question answering typically focus exclusively on text within cells. However, real-world data is inherently multimodal, often blending images such as…
Enhancing Temporal Understanding in LLMs for Semi-structured Tables
Irwin Deng, Kushagra Dixit, Vivek Gupta +1
Temporal reasoning over tabular data presents substantial challenges for large language models (LLMs), as evidenced by recent research. In this study, we conduct a comprehensive an…
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Shubhankar Singh, Purvi Chaurasia, Yerram Varun +4
Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchm…