5 papers
FoveaTree: Information-Density Trees for Query-Guided Visual Token Pruning in Video Large Language Models
Zihan Meng, Weidong Chen, Dexiang Hong +2
Video large language models (Video LLMs) encode each frame into hundreds of visual tokens, making video inference expensive. Existing methods reduce this cost through similarity-ba…
Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioning
Zihan Meng, Dexiang Hong, Weidong Chen +3
Audio-visual captioning generates natural language descriptions from video and audio content. Multimodal LLMs have advanced this task, but both modalities contribute many tokens to…
Agent Learning via Early Experience
Kai Zhang, Xiangchao Chen, Bo Liu +27
A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks. However, training agents fro…
Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video Retrieval
Dohwan Ko, Ji Soo Lee, Minhyuk Choi +2
Text-Video Retrieval aims to find the most relevant text (or video) candidate given a video (or text) query from large-scale online databases. Recent work leverages multi-modal lar…
Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval
Young Kyun Jang, Donghyun Kim, Zihang Meng +2
Composed Image Retrieval (CIR) is a task that retrieves images similar to a query, based on a provided textual modification. Current techniques rely on supervised learning for CIR…