227 citations · 341 across the 26 of their papers we have counts for
26 papers
Reducing Class-Wise Performance Disparity via Margin Regularization
Beier Zhu, Kesen Zhao, Jiequan Cui +4
Deep neural networks often exhibit substantial disparities in class-wise accuracy, even when trained on class-balanced data, posing concerns for reliable deployment. While prior ef…
Real-Time Motion-Controllable Autoregressive Video Diffusion
Kesen Zhao, Jiaxin Shi, Beier Zhu +5
Real-time motion-controllable video generation remains challenging due to the inherent latency of bidirectional diffusion models and the lack of effective autoregressive (AR) appro…
Generalized Visual Relation Detection with Diffusion Models
Kaifeng Gao, Siqi Chen, Hanwang Zhang +3
Visual relation detection (VRD) aims to identify relationships (or interactions) between object pairs in an image. Although recent VRD models have achieved impressive performance,…
Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization
Kesen Zhao, Beier Zhu, Qianru Sun +1
Chain-of-thought (CoT) reasoning greatly improves the interpretability and problem-solving abilities of multimodal large language models (MLLMs). However, existing approaches are f…
Unified Generative and Discriminative Training for Multi-modal Large Language Models
Wei Chow, Juncheng Li, Qifan Yu +7
In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle…
Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration
Kaihang Pan, Zhaoyu Fan, Juncheng Li +6
The swift advancement in Multimodal LLMs (MLLMs) also presents significant challenges for effective knowledge editing. Current methods, including intrinsic knowledge editing and ex…
Few-shot Learner Parameterization by Diffusion Time-steps
Zhongqi Yue, Pan Zhou, Richang Hong +2
Even when using large multi-modal foundation models, few-shot learning is still challenging -- if there is no proper inductive bias, it is nearly impossible to keep the nuanced cla…
Exploring Diffusion Time-steps for Unsupervised Representation Learning
Zhongqi Yue, Jiankun Wang, Qianru Sun +3
Representation learning is all about discovering the hidden modular attributes that generate the data faithfully. We explore the potential of Denoising Diffusion Probabilistic Mode…
Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models
Beier Zhu, Kaihua Tang, Qianru Sun +1
Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Yet, the zero-shot performance is less competitive than a fully supervised o…
Invariant Training 2D-3D Joint Hard Samples for Few-Shot Point Cloud Recognition
Xuanyu Yi, Jiajun Deng, Qianru Sun +3
We tackle the data scarcity challenge in few-shot point cloud recognition of 3D objects by using a joint prediction from a conventional 3D model and a well-trained 2D model. Surpri…
Class-Incremental Exemplar Compression for Class-Incremental Learning
Zilin Luo, Yaoyao Liu, Bernt Schiele +1
Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-…
Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detection
Hui Lv, Zhongqi Yue, Qianru Sun +3
Weakly Supervised Video Anomaly Detection (WSVAD) is challenging because the binary anomaly label is only given on the video level, but the output requires snippet-level prediction…
Semantic Scene Completion with Cleaner Self
Fengyun Wang, Dong Zhang, Hanwang Zhang +2
Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-p…
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
Kaifeng Gao, Long Chen, Hanwang Zhang +2
Prompt tuning with large-scale pretrained vision-language models empowers open-vocabulary predictions trained on limited base categories, e.g., object classification and detection.…
RMM: Reinforced Memory Management for Class-Incremental Learning
Yaoyao Liu, Bernt Schiele, Qianru Sun
Class-Incremental Learning (CIL) [40] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free…
Online Hyperparameter Optimization for Class-Incremental Learning
Yaoyao Liu, Yingying Li, Bernt Schiele +1
Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity…
Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation
Jianqiang Huang, Jian Wang, Qianru Sun +1
Extracting class activation maps (CAM) is a key step for weakly-supervised semantic segmentation (WSSS). The CAM of convolution neural networks fails to capture long-range feature…
Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalization
Jiaxin Qi, Kaihua Tang, Qianru Sun +2
Out-Of-Distribution generalization (OOD) is all about learning invariance against environmental changes. If the context in every class is evenly distributed, OOD would be trivial b…
Equivariance and Invariance Inductive Bias for Learning from Insufficient Data
Tan Wang, Qianru Sun, Sugiri Pranata +2
We are interested in learning robust models from insufficient data, without the need for any externally pre-trained checkpoints. First, compared to sufficient data, we show why ins…
Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation
Zhaozheng Chen, Tan Wang, Xiongwei Wu +3
Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly-supervised semantic segmentation (WSSS). Yet, we find that the crux…
Deconfounded Visual Grounding
Jianqiang Huang, Yu Qin, Jiaxin Qi +2
We focus on the confounding bias between language and location in the visual grounding pipeline, where we find that the bias is the major visual reasoning bottleneck. For example,…
Self-Supervised Learning Disentangled Group Representation as Feature
Tan Wang, Zhongqi Yue, Jianqiang Huang +2
A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In…
Causal Attention for Unbiased Visual Recognition
Tan Wang, Chang Zhou, Qianru Sun +1
Attention module does not always help deep models learn causal features that are robust in any confounding context, e.g., a foreground object feature is invariant to different back…
Transporting Causal Mechanisms for Unsupervised Domain Adaptation
Zhongqi Yue, Qianru Sun, Xian-Sheng Hua +1
Existing Unsupervised Domain Adaptation (UDA) literature adopts the covariate shift and conditional shift assumptions, which essentially encourage models to learn common features a…