4 papers
Memory as Weights: Internalizing Long-Term History for Streaming Videos
Zhuoming Liu, Jinhong Lin, Lin Zhang +2
Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries. Existing memory m…
DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models
Zhuoming Liu, Jinhong Lin, Kwan Man Cheng +3
Many modern vision-language models (VLMs) build on autoregressive decoding of discrete tokens. While text-based output interfaces enable scalable pretraining and strong zero-shot g…
PAVE: Patching and Adapting Video Large Language Models
Zhuoming Liu, Yiquan Li, Khoi Duc Nguyen +2
Pre-trained video large language models (Video LLMs) exhibit remarkable reasoning capabilities, yet adapting these models to new tasks involving additional modalities or data types…
AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning
Yiwu Zhong, Zhuoming Liu, Yin Li +1
Large language models (LLMs) have enabled the creation of multi-modal LLMs that exhibit strong comprehension of visual data such as images and videos. However, these models usually…