9 papers
Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering
Zikui Cai, Kaushal Janga, Tan Dat Dao +15
Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. Ho…
LADBench: A Benchmark for Logical Fault Detection in Images
Sahasra Kondapalli, Lara Radovanovic, Aadi Palnitkar +2
Large Vision Language Models (VLMs) excel at visual question answering and semantic grounding, but their capacity for autonomous logical reasoning remains underexplored. Existing a…
AUDITA: A New Dataset to Audit Humans vs. AI Skill at Audio QA
Tasnim Kabir, Dmytro Kurdydyk, Aadi Palnitkar +3
Existing audio question answering benchmarks largely emphasize sound event classification or caption-grounded queries, often enabling models to succeed through shortcut strategies,…
Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Models
Aadi Palnitkar, Mingyang Mao, Nicholas Waytowich +2
As large language models (LLMs) are applied to increasingly longer and more complex tasks, there is a growing need for realistic long-context benchmarks that require selective read…
Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
Minghui Liu, Aadi Palnitkar, Tahseen Rabbani +9
Large language models (LLMs) have demonstrated remarkable performance on long-context tasks, but are often bottlenecked by memory constraints. Namely, the KV cache, which is used t…
EREBUS: End-to-end Robust Event Based Underwater Simulation
Hitesh Kyatham, Arjun Suresh, Aadi Palnitkar +1
The underwater domain presents a vast array of challenges for roboticists and computer vision researchers alike, such as poor lighting conditions and high dynamic range scenes. In…