13 papers · 1 filter
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
Niu Lian, Yuting Wang, Hanshu Yao +5
While multimodal large language models have demonstrated impressive short-term reasoning, they struggle with long-horizon video understanding due to limited context windows and sta…
Love Me, Love My Label: Rethinking the Role of Labels in Prompt Retrieval for Visual In-Context Learning
Tianci Luo, Haohao Pan, Jinpeng Wang +5
Visual in-context learning (VICL) enables visual foundation models to handle multiple tasks by steering them with demonstrative prompts. The choice of such prompts largely influenc…
Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval
Jun Li, Xuhang Lou, Jinpeng Wang +4
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global…
PromptHub: Enhancing Multi-Prompt Visual In-Context Learning with Locality-Aware Fusion, Concentration and Alignment
Tianci Luo, Jinpeng Wang, Shiyu Qin +5
Visual In-Context Learning (VICL) aims to complete vision tasks by imitating pixel demonstrations. Recent work pioneered prompt fusion that combines the advantages of various demon…
DeAR: Fine-Grained VLM Adaptation by Decomposing Attention Head Roles
Yiming Ma, Hongkun Yang, Lionel Z. Wang +3
Prompt learning is a dominant paradigm for adapting pre-trained Vision-Language Models (VLMs) to downstream tasks. However, existing methods often rely on a simplistic, layer-centr…
CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation
Zhenyu Lu, Liupeng Li, Jinpeng Wang +4
Existing works on reasoning segmentation either connect hidden features from a language model directly to a mask decoder or represent positions in text, which limits interpretabili…