5 papers
SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents
Yibin Huang, Jixiang Hong, Zongzhao Li +8
Latents from vision foundation models (VFMs) are semantically rich and well suited for visual understanding. Recent representation autoencoder methods such as RAE have shown that t…
Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself
Yuhang Dai, Xingyi Yang
Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually…
StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression
Yilong Chen, Xiang Bai, Zhibin Wang +4
Video Large Language Models (Video-LLMs) have demonstrated significant potential in the areas of video captioning, search, and summarization. However, current Video-LLMs still face…
Sparrow: Data-Efficient Video-LLM with Text-to-Image Augmentation
Shukang Yin, Chaoyou Fu, Sirui Zhao +7
Recent years have seen the success of Multimodal Large Language Models (MLLMs) in the domain of vision understanding. The success of these models can largely be attributed to the d…
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Chaoyou Fu, Yuhan Dai, Yongdong Luo +18
In the quest for artificial general intelligence, Multi-modal Large Language Models (MLLMs) have emerged as a focal point in recent advancements. However, the predominant focus rem…