3 citations · 3 across the 6 of their papers we have counts for
7 papers
ReToken: Improving Long-Context VLMs with Visual Retrieval Token
Yao Xiao, Reuben Tan, Zhen Zhu +3
Long visual contexts challenge vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once can exceed GPU memory limits. We p…
AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning
Sarthak Jain, Qiran Hu, Zhen Zhu +1
Multimodal models such as CLIP learn a shared embedding space for cross-modal retrieval, but continual adaptation to sequentially arriving data can disrupt the cross-modal alignmen…
Norm Anchors Make Model Edits Last
Mingda Liu, Zhenghan Zhu, Ze'an Miao +1
Sequential Locate-and-Edit (L&E) model editing can fail abruptly after many edits. We identify and formalize this failure as a positive norm-feedback loop, in which solved value ve…
How to Teach Large Multimodal Models New Skills
Zhen Zhu, Yiming Gong, Yao Xiao +2
How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills while monitoring general ability o…
TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models
Yao Xiao, Qiqian Fu, Heyi Tao +3
Image-text models excel at image-level tasks but struggle with detailed visual understanding. While these models provide strong visual-language alignment, segmentation models like…
Anytime Continual Learning for Open Vocabulary Classification
Zhen Zhu, Yiming Gong, Derek Hoiem
We propose an approach for anytime continual learning (AnytimeCL) for open vocabulary image classification. The AnytimeCL problem aims to break away from batch training and rigid m…
Survey on Memory-Augmented Neural Networks: Cognitive Insights to AI Applications
Savya Khosla, Zhen Zhu, Yifei He
This paper explores Memory-Augmented Neural Networks (MANNs), delving into how they blend human-like memory processes into AI. It covers different memory types, like sensory, short…