9 papers
Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models
Yuanyuan Tian, Wenwen Li, Xiao Chen +4
Semantic search and recommendation of similar documents, such as news and reports about unusual environmental events (e.g., a dead whale washed ashore in Alaska) that contain spati…
MMTABREAL: Real-World Benchmark for Multimodal Table Understanding
Prasham Titiya, Jainil Trivedi, Chitta Baral +1
Multimodal tables i.e. tabular layouts interleaved with charts, maps, icons, and color encodings are ubiquitous in real applications yet remain difficult for Multimodal Large Langu…
Rethinking Information Synthesis in Multimodal Question Answering A Multi-Agent Perspective
Krishna Singh Rajput, Tejas Anvekar, Chitta Baral +1
Recent advances in multimodal question answering have primarily focused on combining heterogeneous modalities or fine-tuning multimodal large language models. While these approache…
ViTaB-A: Evaluating Multimodal Large Language Models on Visual Table Attribution
Yahia Alqurnawi, Preetom Biswas, Anmol Rao +3
Multimodal Large Language Models (mLLMs) are often used to answer questions in structured data such as tables in Markdown, JSON, and images. While these models can often give corre…
The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
Tejas Anvekar, Fenil Bardoliya, Pavan K. Turaga +2
Recent advances in multimodal large language models (MLLMs) have yielded increasingly powerful models, yet their perceptual capacities remain poorly characterized. In practice, mos…
GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning
Shikhhar Siingh, Abhinav Rawat, Chitta Baral +1
Publicly significant images from events hold valuable contextual information, crucial for journalism and education. However, existing methods often struggle to extract this relevan…