24 citations · 74 across the 27 of their papers we have counts for
17 papers · 1 filter
STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
Yiming Liang, Yixiao Chen, Yiyang Zhou +8
Many video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames. Existing methods built on large vision-language models (LVLMs) often a…
MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data
Zongxia Li, Hongyang Du, Chengsong Huang +8
Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention…
Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baseline
Guo Chen, Lidong Lu, Yicheng Liu +17
While datasets for video understanding have scaled to hour-long durations, they typically consist of densely concatenated clips that differ from natural, unscripted daily life. To…
First Frame Is the Place to Go for Video Content Customization
Jingxi Chen, Zongxia Li, Zhichao Liu +6
What role does the first frame play in video generation models? Traditionally, it's viewed as the spatial-temporal starting point of a video, merely a seed for subsequent animation…
Self-Rewarding Vision-Language Model via Reasoning Decomposition
Zongxia Li, Wenhao Yu, Chengsong Huang +8
Vision-Language Models (VLMs) often suffer from visual hallucinations: generating things that are not consistent with visual inputs and language shortcuts, where they skip the visu…
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding
Zongxia Li, Xiyang Wu, Guangyao Shi +6
Vision-Language Models (VLMs) have achieved strong results in video understanding, yet a key question remains: do they truly comprehend visual content or only learn shallow correla…
ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
Yijun Liang, Ming Li, Chenrui Fan +7
Color plays an important role in human perception and usually provides critical clues in visual reasoning. However, it is unclear whether and how vision-language models (VLMs) can…
AIDE: Agentically Improve Visual Language Model with Domain Experts
Ming-Chang Chiu, Fuxiao Liu, Karan Sapra +5
The enhancement of Visual Language Models (VLMs) has traditionally relied on knowledge distillation from larger, more capable models. This dependence creates a fundamental bottlene…
A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
Zongxia Li, Xiyang Wu, Hongyang Du +3
Large vision-language models (VLMs) have evolved rapidly from contrastive image-text encoders and adapter-based assistants into natively multimodal foundation models that support l…
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
Min Shi, Fuxiao Liu, Shihao Wang +13
The ability to accurately interpret complex visual information is a crucial topic of multimodal large language models (MLLMs). Recent work indicates that enhanced visual perception…
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences
Xiyao Wang, Yuhang Zhou, Xiaoyu Liu +9
Multimodal Large Language Models (MLLMs) have demonstrated proficiency in handling a variety of visual-language tasks. However, current MLLM benchmarks are predominantly designed t…
HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
Tianrui Guan, Fuxiao Liu, Xiyang Wu +9
We introduce HallusionBench, a comprehensive benchmark designed for the evaluation of image-context reasoning. This benchmark presents significant challenges to advanced large visu…
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
Fuxiao Liu, Kevin Lin, Linjie Li +3
Despite the promising progress in multi-modal tasks, current large multi-modal models (LMMs) are prone to hallucinating inconsistent descriptions with respect to the associated ima…
DocumentCLIP: Linking Figures and Main Body Text in Reflowed Documents
Fuxiao Liu, Hao Tan, Chris Tensmeyer
Vision-language pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text. While existing vision-…
SCP: Soft Conditional Prompt Learning for Aerial Video Action Recognition
Xijun Wang, Ruiqi Xian, Tianrui Guan +2
We present a new learning approach, Soft Conditional Prompt Learning (SCP), which leverages the strengths of prompt learning for aerial video action recognition. Our approach is de…
COVID-VTS: Fact Extraction and Verification on Short Video Platforms
Fuxiao Liu, Yaser Yacoob, Abhinav Shrivastava
We introduce a new benchmark, COVID-VTS, for fact-checking multi-modal information involving short-duration videos with COVID19- focused information from both the real world and ma…
Visual News: Benchmark and Challenges in News Image Captioning
Fuxiao Liu, Yinghan Wang, Tianlu Wang +1
We propose Visual News Captioner, an entity-aware model for the task of news image captioning. We also introduce Visual News, a large-scale benchmark consisting of more than one mi…