1 citations · 2 across the 51 of their papers we have counts for
24 papers · 1 filter
What Should a Streaming Video Model Remember?
Haonan Ge, Yiwei Wang, Hang Wu +1
Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation bu…
Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding
Hang Wu, Sherin Mary Mathews, Yujun Cai +2
Online streaming video understanding requires models to process continuous visual inputs and respond to user queries in real time, where the unbounded stream and unpredictable quer…
Hard to Read, Easy to Jailbreak: How Visual Degradation Bypasses MLLM Safety Alignment
Zhixue Song, Boyan Han, Yiwei Wang +1
Recent advancements in visual context compression enable MLLMs to process ultra-long contexts efficiently by rendering text into images. However, we identify a critical vulnerabili…
VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Honghao Fu, Miao Xu, Yiwei Wang +3
Scaling multimodal large language models (MLLMs) to long videos is constrained by limited context windows. While retrieval-augmented generation (RAG) is a promising remedy by organ…
Hierarchical, Interpretable, Label-Free Concept Bottleneck Model
Haodong Xie, Yujun Cai, Rahul Singh Maharjan +3
Concept Bottleneck Models (CBMs) introduce interpretability to black-box deep learning models by predicting labels through human-understandable concepts. However, unlike humans, wh…
Finding Distributed Object-Centric Properties in Self-Supervised Transformers
Samyak Rawlekar, Amitabh Swain, Yujun Cai +3
Self-supervised Vision Transformers (ViTs) like DINO show an emergent ability to discover objects, typically observed in [CLS] token attention maps of the final layer. However, the…
CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoning
Hang Wu, Yujun Cai, Zehao Li +4
Understanding camera dynamics is a fundamental pillar of video spatial intelligence. However, existing multimodal models predominantly treat this task as a black-box classification…
Bi-Erasing: A Bidirectional Framework for Concept Removal in Diffusion Models
Hao Chen, Yiwei Wang, Songze Li
Concept erasure, which fine-tunes diffusion models to remove undesired or harmful visual concepts, has become a mainstream approach to mitigating unsafe or illegal image generation…
EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound Generation
Bingxuan Li, Yiming Cui, Yicheng He +4
Sound effects build an essential layer of multimodal storytelling, shaping the emotional atmosphere and the narrative semantics of videos. Despite recent advancement in video-text-…
PAS: A Training-Free Stabilizer for Temporal Encoding in Video LLMs
Bowen Sun, Yujun Cai, Ming-Hsuan Yang +2
Video LLMs suffer from temporal inconsistency: small shifts in frame timing can flip attention and suppress relevant frames. We trace this instability to the common extension of Ro…
Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
Sifan Li, Hongkai Chen, Yujun Cai +4
Vision Language Models (VLMs) have achieved impressive progress in multimodal reasoning; yet, they remain vulnerable to hallucinations, where outputs are not grounded in visual evi…
FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
Haonan Ge, Yiwei Wang, Kai-Wei Chang +2
Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each questio…
Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Tools
Zhenlong Yuan, Xiangyan Qu, Chengxuan Qian +8
Multimodal large language models (MLLMs) have demonstrated remarkable potential in bridging visual and textual reasoning, yet their reliance on text-centric priors often limits the…
ChainMPQ: Interleaved Text-Image Reasoning Chains for Mitigating Relation Hallucinations
Yike Wu, Yiwei Wang, Yujun Cai
While Large Vision-Language Models (LVLMs) achieve strong performance in multimodal tasks, hallucinations continue to hinder their reliability. Among the three categories of halluc…
Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding
Zhecheng Li, Guoxian Song, Yiwei Wang +3
Grounding natural language queries in graphical user interfaces (GUIs) presents a challenging task that requires models to comprehend diverse UI elements across various application…
Cure or Poison? Embedding Instructions Visually Alters Hallucination in Vision-Language Models
Zhaochen Wang, Yiwei Wang, Yujun Cai
Vision-Language Models (VLMs) often suffer from hallucination, partly due to challenges in aligning multimodal information. We propose Prompt-in-Image, a simple method that embeds…
MRFD: Multi-Region Fusion Decoding with Self-Consistency for Mitigating Hallucinations in LVLMs
Haonan Ge, Yiwei Wang, Ming-Hsuan Yang +1
Large Vision-Language Models (LVLMs) have shown strong performance across multimodal tasks. However, they often produce hallucinations -- text that is inconsistent with visual inpu…
: Advancing Img2LaTeX Conversion via Visual Reasoning with Attention-Guided Refinement
Zhecheng Li, Guoxian Song, Yiwei Wang +3
Img2LaTeX is a practically important task that involves translating mathematical expressions and structured visual content from images into LaTeX code. In recent years, vision-lang…
Toward Rich Video Human-Motion2D Generation
Ruihao Xi, Xuekuan Wang, Yongcheng Li +5
Generating realistic and controllable human motions, particularly those involving rich multi-character interactions, remains a significant challenge due to data scarcity and the co…
Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models
Zhaochen Wang, Bryan Hooi, Yiwei Wang +3
Vision-language models (VLMs) have advanced rapidly in processing multimodal information, but their ability to reconcile conflicting signals across modalities remains underexplored…
How does Watermarking Affect Visual Language Models in Document Understanding?
Chunxue Xu, Yiwei Wang, Bryan Hooi +2
Visual Language Models (VLMs) have become foundational models for document understanding tasks, widely used in the processing of complex multimodal documents across domains such as…
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization
Shuyang Hao, Yiwei Wang, Bryan Hooi +4
In the realm of large vision-language models (LVLMs), adversarial jailbreak attacks serve as a red-teaming approach to identify safety vulnerabilities of these models and their ass…
Lost in Edits? A -Compass for AIGC Provenance
Wenhao You, Bryan Hooi, Yiwei Wang +5
Recent advancements in diffusion models have driven the growth of text-guided image editing tools, enabling precise and iterative modifications of synthesized content. However, as…
Efficiently Disentangling CLIP for Multi-Object Perception
Samyak Rawlekar, Yujun Cai, Yiwei Wang +2
Vision-language models like CLIP excel at recognizing the single, prominent object in a scene. However, they struggle in complex scenes containing multiple objects. We identify a f…