5 papers
Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners
Yunhan Wang, Eshika Khandelwal, Edson Araujo +3
Multimodal Large Language Models (MLLMs) have demonstrated remarkable abilities when analyzing images, yet translating these capabilities to few-shot image classification remains c…
FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs
Eshika Khandelwal, Jingjing Pan, Mingfang Zhang +3
Multimodal large language models (MLLMs) are predominantly evaluated on free-form vision-language tasks such as visual question answering, captioning, and summarization. However, t…
More than a Moment: Towards Coherent Sequences of Audio Descriptions
Eshika Khandelwal, Junyu Xie, Tengda Han +5
Audio Descriptions (ADs) convey essential on-screen information, allowing visually impaired audiences to follow videos. To be effective, ADs must form a coherent sequence that help…
What You See is What You Ask: Evaluating Audio Descriptions
Divy Kala, Eshika Khandelwal, Makarand Tapaswi
Audio descriptions (ADs) narrate important visual details in movies, enabling Blind and Low Vision (BLV) users to understand narratives and appreciate visual details. Existing work…
Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation
Junyu Xie, Tengda Han, Max Bain +5
Our objective is the automatic generation of Audio Descriptions (ADs) for edited video material, such as movies and TV series. To achieve this, we propose a two-stage framework tha…