17 citations · 29 across the 12 of their papers we have counts for
13 papers
Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning
Cheng Ye, Weidong Chen, Zhaobo Qi +2
While multimodal large language models (MLLMs) have demonstrated exceptional capabilities in objective understanding tasks, their performance in affective reasoning still falls sig…
Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
Sirui Chen, Lei Xu, Yuying Zhao +6
Recent RL methods have substantially improved the reasoning abilities of LLMs. Existing reward designs mainly follow two paradigms: (1) Reinforcement learning with verifiable rewar…
Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization
Kesen Zhao, Beier Zhu, Qianru Sun +1
Chain-of-thought (CoT) reasoning greatly improves the interpretability and problem-solving abilities of multimodal large language models (MLLMs). However, existing approaches are f…
Robust Fine-tuning of Zero-shot Models via Variance Reduction
Beier Zhu, Jiequan Cui, Hanwang Zhang
When fine-tuning zero-shot models like CLIP, our desideratum is for the fine-tuned model to excel in both in-distribution (ID) and out-of-distribution (OOD). Recently, ensemble-bas…
Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting
Xingyu Zhu, Beier Zhu, Yi Tan +3
Vision-language models, such as CLIP, have shown impressive generalization capacities when using appropriate text descriptions. While optimizing prompts on downstream labeled data…
Selective Vision-Language Subspace Projection for Few-shot CLIP
Xingyu Zhu, Beier Zhu, Yi Tan +3
Vision-language models such as CLIP are capable of mapping the different modality data into a unified feature space, enabling zero/few-shot inference by measuring the similarity of…
Classes Are Not Equal: An Empirical Study on Image Recognition Fairness
Jiequan Cui, Beier Zhu, Xin Wen +3
In this paper, we present an empirical study on image recognition fairness, i.e., extreme class accuracy disparity on balanced data like ImageNet. We experimentally demonstrate tha…
Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models
Beier Zhu, Kaihua Tang, Qianru Sun +1
Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Yet, the zero-shot performance is less competitive than a fully supervised o…
Debiased Fine-Tuning for Vision-language Models by Prompt Regularization
Beier Zhu, Yulei Niu, Saeil Lee +2
We present a new paradigm for fine-tuning large-scale visionlanguage pre-trained models on downstream task, dubbed Prompt Regularization (ProReg). Different from traditional fine-t…
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
Chen Chen, Yuchen Hu, Qiang Zhang +3
Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition. Mainstream methods focus on fusing audio and visua…
Prompt-aligned Gradient for Prompt Tuning
Beier Zhu, Yulei Niu, Yucheng Han +2
Thanks to the large pre-trained vision-language models (VLMs) like CLIP, we can craft a zero-shot classifier by "prompt", e.g., the confidence score of an image being "[CLASS]" can…
Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification
Beier Zhu, Yulei Niu, Xian-Sheng Hua +1
We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of tail…
Fast and Accurate: Structure Coherence Component for Face Alignment
Beier Zhu, Chunze Lin, Quan Wang +2
In this paper, we propose a fast and accurate coordinate regression method for face alignment. Unlike most existing facial landmark regression methods which usually employ fully co…