11 papers · 1 filter
TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction
Nikos Athanasiou, Ilya A. Petrov, Angela Yao +8
Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal reasoning pipelines. However, r…
LightAVSeg: Lightweight Audio-Visual Segmentation
Qing Zhong, Guodong Ding, Lingqiao Liu +3
Audio-Visual Segmentation (AVS) targets pixel level localization of sounding emitting objects in videos. However, existing models rely on dense cross-modal attention with quadratic…
Gate-and-Merge: Zero-shot Compositional Personalization of Vision Language Models
Guodong Ding, Angela Yao
This paper tackles compositional personalization of vision-language models (VLMs). In this problem, multiple user-defined concepts must be recognized or described jointly at test t…
DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving
Xiaolu Liu, Yicong Li, Song Wang +3
Recently, world models have been incorporated into the autonomous driving systems to improve the planning reliability. Existing approaches typically predict future states through a…
On Discriminative vs. Generative classifiers: Rethinking MLLMs for Action Understanding
Zhanzhong Pang, Dibyadip Chatterjee, Fadime Sener +1
Multimodal Large Language Models (MLLMs) have advanced open-world action understanding and can be adapted as generative classifiers for closed-set settings by autoregressively gene…
Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understanding
Dibyadip Chatterjee, Edoardo Remelli, Yale Song +9
We introduce ProVideLLM, an end-to-end framework for real-time procedural video understanding. ProVideLLM integrates a multimodal cache configured to store two types of tokens - ve…