2 citations · 2 across the 32 of their papers we have counts for
26 papers · 1 filter
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
Xingyu Zhu, Huanshen Wu, Shuo Wang +4
Pre-trained Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under adversarial perturbations. Existing tes…
From Uncertainty to Stability and Fidelity: Guiding Sparse-View 3D Gaussian Splatting with Fisher Information
Junbao Zhou, Qingshan Xu, Yuan Zhou +7
3D Gaussian Splatting (3DGS) has emerged as a promising technique for novel view synthesis. However, 3DGS requires dense input views to achieve high-quality rendering. In sparse-vi…
Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation
Xingyu Zhu, Junfeng Fang, Shuo Wang +4
Large Vision-Language Models (LVLMs) exhibit powerful generative capabilities but frequently produce hallucinations that compromise output reliability. Fine-tuning on annotated dat…
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Sudong Wang, Weiquan Huang, Xiaomin Yu +9
The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiab…
Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models
Xingyu Zhu, Beier Zhu, Shuo Wang +4
As vision-language models (VLMs) are increasingly deployed in open-world scenarios, they can be easily induced by visual jailbreak attacks to generate harmful content, posing serio…
Adapting Point Cloud Analysis via Multimodal Bayesian Distribution Learning
Xingyu Zhu, Liang Yi, Shuo Wang +4
Multimodal 3D vision-language models show strong generalization across diverse 3D tasks, but their performance still degrades notably under domain shifts. This has motivated recent…
Few-Step Diffusion Sampling Through Instance-Aware Discretizations
Liangyu Yuan, Ruoyu Wang, Tong Zhao +4
Diffusion and flow matching models generate high-fidelity data by simulating paths defined by Ordinary or Stochastic Differential Equations (ODEs/SDEs), starting from a tractable p…
DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling
Tong Zhao, Mingkun Lei, Liangyu Yuan +5
Diffusion Models (DMs) have achieved state-of-the-art generative performance across multiple modalities, yet their sampling process remains prohibitively slow due to the need for h…
NOUS: Video-Driven 3D Human Reaction Generation via Observation-Reaction Mutual Steering
Yuan Zhou, Luanyuan Dai, Yongzhi Li +7
Video-driven 3D human reaction generation aims to synthesize 3D human motion in response to the action observed in a video, playing an important role in interactive multimedia syst…
Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation
Xingyu Zhu, Kesen Zhao, Liang Yi +4
Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language reasoning, yet they remain vulnerable to hallucination, where generated content deviat…
GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
Xingyu Zhu, Beier Zhu, Junfeng Fang +4
Large vision-language models (LVLMs) have achieved remarkable progress in vision-language reasoning tasks, yet ensuring their safety remains a critical challenge. Recent input-side…
Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought
Kesen Zhao, Beier Zhu, Junbao Zhou +3
Recent MLLMs increasingly rely on visual chain-of-thought to perform region-grounded reasoning over images. However, existing approaches ground regions via either textified coordin…
Free Lunch for Stabilizing Rectified Flow Inversion
Chenru Wang, Beier Zhu, Chi Zhang
Rectified-Flow (RF)-based generative models have recently emerged as strong alternatives to traditional diffusion models, demonstrating state-of-the-art performance across various…
Parallel Diffusion Solver via Residual Dirichlet Policy Optimization
Ruoyu Wang, Ziyu Li, Beier Zhu +5
Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature. Existing solver-based a…
Hierarchical Semantic Alignment for Image Clustering
Xingyu Zhu, Beier Zhu, Yunfan Li +4
Image clustering is a classic problem in computer vision, which categorizes images into different groups. Recent studies utilize nouns as external semantic knowledge to improve clu…
Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling
Ruoyu Wang, Beier Zhu, Junzhi Li +2
Diffusion-based generative processes, formulated as differential equation solving, frequently balance computational speed with sample quality. Our theoretical investigation of ODE-…
Enhancing CLIP Robustness via Cross-Modality Alignment
Xingyu Zhu, Beier Zhu, Shuo Wang +2
Vision-language models (VLMs) such as CLIP demonstrate strong generalization in zero-shot classification but remain highly vulnerable to adversarial perturbations. Existing methods…
Real-Time Motion-Controllable Autoregressive Video Diffusion
Kesen Zhao, Jiaxin Shi, Beier Zhu +5
Real-time motion-controllable video generation remains challenging due to the inherent latency of bidirectional diffusion models and the lack of effective autoregressive (AR) appro…
Streaming Drag-Oriented Interactive Video Manipulation: Drag Anything, Anytime!
Junbao Zhou, Yuan Zhou, Kesen Zhao +4
Achieving streaming, fine-grained control over the outputs of autoregressive video diffusion models remains challenging, making it difficult to ensure that they consistently align…
Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models
Beier Zhu, Ruoyu Wang, Tong Zhao +2
Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature. Existing solver-based a…
Detail++: Training-Free Detail Enhancer for T2I Diffusion Models
Lifeng Chen, Jiner Wang, Zihao Pan +3
Recent advances in text-to-image (T2I) generation have led to impressive visual results. However, these models still face significant challenges when handling complex prompt, parti…
CoDi: Subject-Consistent and Pose-Diverse Text-to-Image Generation
Zhanxin Gao, Beier Zhu, Liang Yao +2
Subject-consistent generation (SCG)-aiming to maintain a consistent subject identity across diverse scenes-remains a challenge for text-to-image (T2I) models. Existing training-fre…
Dynamic Multimodal Prototype Learning in Vision-Language Models
Xingyu Zhu, Shuo Wang, Beier Zhu +6
With the increasing attention to pre-trained vision-language models (VLMs), \eg, CLIP, substantial efforts have been devoted to many downstream tasks, especially in test-time adapt…
Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness
Beier Zhu, Jiequan Cui, Hanwang Zhang +1
While image-text foundation models have succeeded across diverse downstream tasks, they still face challenges in the presence of spurious correlations between the input and label.…