Xudong Wang

Bifocal: Training-Free Mixed Granularity Visual Token Pruning for Multimodal LLMs

Peiran Yin, Hao Wu, Zihao Yang +6

Visual token pruning for multimodal large language models (MLLMs) appears highly effective, with existing methods retaining over 95% of unpruned performance on common VQA benchmark…

UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking

Hao Wu, Xudong Wang, Jialiang Zhang +5

One-stream Transformer-based trackers achieve advanced performance in visual object tracking but suffer from significant computational overhead that hinders real-time deployment. W…

ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attention

Wenjie Liu, Hao Wu, Xin Qiu +6

Modern multimodal large language models (MLLMs) adopt a unified self-attention design that processes visual and textual tokens at every Transformer layer, incurring substantial com…

Visually Prompted Benchmarks Are Surprisingly Fragile

Haiwen Feng, Long Lian, Lisa Dunlap +6

A key challenge in evaluating VLMs is testing models' ability to analyze visual content independently from their textual priors. Recent benchmarks such as BLINK probe visual percep…

UnSAMv2: Self-Supervised Learning Enables Segment Anything at Any Granularity

Junwei Yu, Trevor Darrell, XuDong Wang

The Segment Anything Model (SAM) family has become a widely adopted vision foundation model, but its ability to control segmentation granularity remains limited. Users often need t…

Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens

Yiming Qin, Bomin Wei, Jiaxin Ge +4

Vision-Language Models (VLMs) excel at reasoning in linguistic space but struggle with perceptual understanding that requires dense visual perception, e.g., spatial reasoning and g…

Constantly Improving Image Models Need Constantly Improving Benchmarks

Jiaxin Ge, Grace Luo, Heekyung Lee +7

Recent advances in image generation, often driven by proprietary systems like GPT-4o Image Gen, regularly introduce new capabilities that reshape how users interact with these mode…

Reconstruction Alignment Improves Unified Multimodal Models

Ji Xie, Trevor Darrell, Luke Zettlemoyer +1

Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) wh…

TULIP: Towards Unified Language-Image Pretraining

Zineng Tang, Long Lian, Seun Eisape +6

Despite the recent success of image-text contrastive models like CLIP and SigLIP, these models often struggle with vision-centric tasks that demand high-fidelity image understandin…

Visual Lexicon: Rich Image Features in Language Space

XuDong Wang, Xingyi Zhou, Alireza Fathi +2

We present Visual Lexicon, a novel visual language that encodes rich image information into the text space of vocabulary tokens while retaining intricate visual details that are of…

SegLLM: Multi-round Reasoning Segmentation

XuDong Wang, Shaolun Zhang, Shufan Li +5

We present SegLLM, a novel multi-round interactive reasoning segmentation model that enhances LLM-based segmentation by exploiting conversational memory of both visual and textual…

Segment Anything without Supervision

XuDong Wang, Jingfeng Yang, Trevor Darrell

The Segmentation Anything Model (SAM) requires labor-intensive data labeling. We present Unsupervised SAM (UnSAM) for promptable and automatic whole-image segmentation that does no…