2 citations · 10 across the 21 of their papers we have counts for
21 papers
SAGE: Structure Aware Graph Expansion for Retrieval of Heterogeneous Data
Prasham Titiya, Rohit Khoja, Tomer Wolfson +2
Retrieval-augmented question answering over heterogeneous corpora requires connected evidence across text, tables, and graph nodes. While entity-level knowledge graphs support stru…
Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know"
Dhruv Madhwal, Lyuxin David Zhang, Dan Roth +2
Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations. While decomposed prompting is typic…
CORE-T: COherent REtrieval of Tables for Text-to-SQL
Hassan Soliman, Vivek Gupta, Dan Roth +1
Realistic text-to-SQL workflows often require joining multiple tables. As a result, accurately retrieving the relevant set of tables becomes a key bottleneck for end-to-end perform…
MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents
Tomer Wolfson, Harsh Trivedi, Mor Geva +5
Automated agents, powered by Large language models (LLMs), are emerging as the go-to tool for querying information. However, evaluation benchmarks for LLM agents rarely feature nat…
InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information
Anirudh Iyengar Kaniyar Narayana Iyengar, Srija Mukhopadhyay, Adnan Qidwai +3
We introduce InterChart, a diagnostic benchmark that evaluates how well vision-language models (VLMs) reason across multiple related charts, a task central to real-world applicatio…
PRAISE: Enhancing Product Descriptions with LLM-Driven Structured Insights
Adnan Qidwai, Srija Mukhopadhyay, Prerana Khatiwada +2
Accurate and complete product descriptions are crucial for e-commerce, yet seller-provided information often falls short. Customer reviews offer valuable details but are laborious…
LLM-Symbolic Integration for Robust Temporal Tabular Reasoning
Atharv Kulkarni, Kushagra Dixit, Vivek Srikumar +2
Temporal tabular question answering presents a significant challenge for Large Language Models (LLMs), requiring robust reasoning over structured data, which is a task where tradit…
No Universal Prompt: Unifying Reasoning through Adaptive Prompting for Temporal Table Reasoning
Abhishek Rajgaria, Kushagra Dixit, Mayank Vyas +3
Temporal Table Reasoning is a critical challenge for Large Language Models (LLMs), requiring effective reasoning to extract relevant insights. Despite existence of multiple prompti…
UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification
Poojah Ganesan, Rajat Aayush Jha, Dan Roth +1
Recent advances in large language models (LLMs) have greatly improved Text-to-SQL performance for single-table queries. But, it remains challenging in multi-table databases due to…
Weaver: Interweaving SQL and LLM for Table Reasoning
Rohit Khoja, Devanshu Gupta, Yanjie Fu +2
Querying tables with unstructured data is challenging due to the presence of text (or image), either embedded in the table or in external paragraphs, which traditional SQL struggle…
Leveraging LLM For Synchronizing Information Across Multilingual Tables
Siddharth Khincha, Tushar Kataria, Ankita Anand +2
The vast amount of online information today poses challenges for non-English speakers, as much of it is concentrated in high-resource languages such as English and French. Wikipedi…
TransientTables: Evaluating LLMs' Reasoning on Temporally Evolving Semi-structured Tables
Abhilash Shankarampeta, Harsh Mahajan, Tushar Kataria +2
Humans continuously make new discoveries, and understanding temporal sequence of events leading to these breakthroughs is essential for advancing science and society. This ability…
MAPWise: Evaluating Vision-Language Models for Advanced Map Queries
Srija Mukhopadhyay, Abhishek Rajgaria, Prerana Khatiwada +2
Vision-language models (VLMs) excel at tasks requiring joint understanding of visual and linguistic information. A particularly promising yet under-explored application for these m…
Knowledge-Aware Reasoning over Multimodal Semi-structured Tables
Suyash Vardhan Mathur, Jainit Sushil Bafna, Kunal Kartik +5
Existing datasets for tabular question answering typically focus exclusively on text within cells. However, real-world data is inherently multimodal, often blending images such as…
Enhancing Temporal Understanding in LLMs for Semi-structured Tables
Irwin Deng, Kushagra Dixit, Vivek Gupta +1
Temporal reasoning over tabular data presents substantial challenges for large language models (LLMs), as evidenced by recent research. In this study, we conduct a comprehensive an…
Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness
Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella +3
Chart question answering (CQA) is a crucial area of Visual Language Understanding. However, the robustness and consistency of current Visual Language Models (VLMs) in this field re…
NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models
Pranshu Pandya, Vatsal Gupta, Agney S Talwarr +3
Cognitive textual and visual reasoning tasks, including puzzles, series, and analogies, demand the ability to quickly reason, decipher, and evaluate patterns both textually and spa…
H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables
Nikhil Abhyankar, Vivek Gupta, Dan Roth +1
Tabular reasoning involves interpreting natural language queries about tabular data, which presents a unique challenge of combining language understanding with structured data anal…
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Shubhankar Singh, Purvi Chaurasia, Yerram Varun +4
Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchm…
Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering
Pragya Srivastava, Manuj Malik, Vivek Gupta +2
Large Language Models (LLMs), excel in natural language understanding, but their capability for complex mathematical reasoning with an amalgamation of structured tables and unstruc…
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
Vatsal Gupta, Pranshu Pandya, Tushar Kataria +2
Language models, characterized by their black-box nature, often hallucinate and display sensitivity to input perturbations, causing concerns about trust. To enhance trust, it is im…