2 citations · 8 across the 30 of their papers we have counts for
21 papers · 1 filter
How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective
Janet Wang, Yunbei Zhang, Xiao Wang +1
Medical Vision-Language Models (VLMs) show significant promise for clinical image understanding, offering accurate diagnosis with interpretable reasoning. However, a critical perfo…
Dynamic Hub-and-Spoke Memory for Streaming Video Understanding
Xinru Jiang, Lin Zhao, Xi Xiao +7
Streaming video understanding requires answering questions at arbitrary times over a continuously growing visual stream. The central challenge is to compactly remember long-range h…
Towards Continual Test-Time Adaptation of Vision-Language Models in Open-Vocabulary Semantic Segmentation
Chandler Timm C. Doloriel, Yunbei Zhang, Sarthak Kumar Maharana +5
Open-vocabulary semantic segmentation (OVSS) relies on vision-language alignment to recognize arbitrary text-defined categories, yet this alignment is fragile under continual test-…
Continual Test-Time Adaptation via Entropy Sensitivity-Guidance in Strict Online Setting
Chandler Timm C. Doloriel, Yunbei Zhang, Muhammad Salman Siddiqui +4
Test-time adaptation (TTA) promises robustness under distribution shift by updating a pretrained model on unlabeled test data, but strict online TTA with batch size one and no acce…
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
Sarthak Kumar Maharana, Shambhavi Mishra, Yunbei Zhang +6
Deep neural nets achieve remarkable performance when training and test data share the same distribution, but this assumption frequently breaks in real-world deployment, where data…
Adapting Vision Foundation Models with Cascaded Semantics
Xi Xiao, Xingjian Li, Cheng Han +8
Prompt tuning, a leading parameter-efficient adaptation paradigm in NLP, has recently been extended to computer vision. Visual prompt tuning (VPT) adapts pre-trained vision transfo…
Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation
Yuqi Li, Xi Xiao, Yunbei Zhang +6
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter-efficient adaptation a central problem. Prompt-based adapta…
Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs
Xi Xiao, Chen Liu, Chih-Ting Liao +9
Multimodal large language models (MLLMs) extend large language models (LLMs) with visual perception, enabling joint reasoning over images and text. Despite inheriting strong reason…
Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models
Xi Xiao, Xingjian Li, Yunbei Zhang +7
Visual prompt tuning has emerged as a parameter-efficient fine-tuning approach for adapting large-scale Vision Transformers (ViTs) to downstream tasks. As its learnable prompts are…
Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation
Xi Xiao, Chenrui Ma, Yunbei Zhang +7
Low-Rank Adaptation (LoRA) has become a cornerstone of parameter-efficient fine-tuning (PEFT). Yet, its efficacy is hampered by two fundamental limitations: semantic drift, by trea…
Prime Once, then Reprogram Locally: An Efficient Alternative to Black-Box Service Model Adaptation
Yunbei Zhang, Chengyi Cai, Feng Liu +1
Adapting closed-box service models (i.e., APIs) for target tasks typically relies on reprogramming via Zeroth-Order Optimization (ZOO). However, this standard strategy is known for…
Wan-R1: Verifiable-Reinforcement Learning for Video Reasoning
Ming Liu, Yunbei Zhang, Shilong Liu +2
Video generation models produce visually coherent content but struggle with tasks requiring spatial reasoning and multi-step planning. Reinforcement learning (RL) offers a path to…
Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness
Xin Hu, Haomiao Ni, Yunbei Zhang +3
Vision language models (VLMs) have achieved remarkable success in broad visual understanding, yet they remain challenged by object-centric reasoning on rare objects due to the scar…
Prompt-based Adaptation in Large-scale Vision Models: A Survey
Xi Xiao, Yunbei Zhang, Lin Zhao +12
In computer vision, Visual Prompting (VP) and Visual Prompt Tuning (VPT) have recently emerged as lightweight and effective alternatives to full fine-tuning for adapting large-scal…
Visual Instance-aware Prompt Tuning
Xi Xiao, Yunbei Zhang, Xingjian Li +5
Visual Prompt Tuning (VPT) has emerged as a parameter-efficient fine-tuning paradigm for vision transformers, with conventional approaches utilizing dataset-level prompts that rema…
Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback
Janet Wang, Yunbei Zhang, Zhengming Ding +1
Paucity of medical data severely limits the generalizability of diagnostic ML models, as the full spectrum of disease variability can not be represented by a small clinical dataset…
Describe Anything in Medical Images
Xi Xiao, Yunbei Zhang, Thanh-Huy Nguyen +10
Localized image captioning has made significant progress with models like the Describe Anything Model (DAM), which can generate detailed region-specific descriptions without explic…
Visual Variational Autoencoder Prompt Tuning
Xi Xiao, Yunbei Zhang, Yanshuh Li +5
Parameter-efficient fine-tuning (PEFT) has emerged as a crucial approach for adapting large vision transformers to downstream tasks without the prohibitive computational costs of f…
OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation
Yunbei Zhang, Akshay Mehra, Jihun Hamm
Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations, but their performance is compromised when applied to unseen domains. Previous meth…
On the Fly Neural Style Smoothing for Risk-Averse Domain Generalization
Akshay Mehra, Yunbei Zhang, Bhavya Kailkhura +1
Achieving high accuracy on data from domains unseen during training is a fundamental challenge in domain generalization (DG). While state-of-the-art DG classifiers have demonstrate…
Achieving Reliable and Fair Skin Lesion Diagnosis via Unsupervised Domain Adaptation
Janet Wang, Yunbei Zhang, Zhengming Ding +1
The development of reliable and fair diagnostic systems is often constrained by the scarcity of labeled data. To address this challenge, our work explores the feasibility of unsupe…