592 citations · 3.2k across the 82 of their papers we have counts for
83 papers
The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
Wencheng Ye, Yi Bin, Yujuan Ding +7
Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening eviden…
BadCM: Invisible Backdoor Attack Against Cross-Modal Learning
Zheng Zhang, Xu Yuan, Lei Zhu +2
Despite remarkable successes in unimodal learning tasks, backdoor attacks against cross-modal learning are still underexplored due to the limited generalization and inferior stealt…
Towards Redundancy-Free Sub-networks in Continual Learning
Cheng Chen, Jingkuan Song, LianLi Gao +1
Catastrophic Forgetting (CF) is a prominent issue in continual learning. Parameter isolation addresses this challenge by masking a sub-network for each task to mitigate interferenc…
CUCL: Codebook for Unsupervised Continual Learning
Chen Cheng, Jingkuan Song, Xiaosu Zhu +3
The focus of this study is on Unsupervised Continual Learning (UCL), as it presents an alternative to Supervised Continual Learning which needs high-quality manual labeled data. Th…
Continual Referring Expression Comprehension via Dual Modular Memorization
Heng Tao Shen, Cheng Chen, Peng Wang +3
Referring Expression Comprehension (REC) aims to localize an image region of a given object described by a natural-language expression. While promising performance has been demonst…
Class Gradient Projection For Continual Learning
Cheng Chen, Ji Zhang, Jingkuan Song +1
Catastrophic forgetting is one of the most critical challenges in Continual Learning (CL). Recent approaches tackle this problem by projecting the gradient update orthogonal to the…
Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection
Tao He, Lianli Gao, Jingkuan Song +1
Scene graph generation (SGG) and human-object interaction (HOI) detection are two important visual tasks aiming at localising and recognising relationships between objects, and int…
CHAIN: Exploring Global-Local Spatio-Temporal Information for Improved Self-Supervised Video Hashing
Rukai Wei, Yu Liu, Jingkuan Song +3
Compressing videos into binary codes can improve retrieval speed and reduce storage overhead. However, learning accurate hash codes for video retrieval can be challenging due to hi…
X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention
Yixuan Zhou, Xuanhan Wang, Xing Xu +2
High-resolution representation is necessary for human pose estimation to achieve high performance, and the ensuing problem is high computational complexity. In particular, predomin…
Part-Aware Transformer for Generalizable Person Re-identification
Hao Ni, Yuke Li, Lianli Gao +2
Domain generalization person re-identification (DG-ReID) aims to train a model on source domains and generalize well on unseen domains. Vision Transformer usually yields better gen…
Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval
Hao Li, Jingkuan Song, Lianli Gao +2
Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are ofte…
DePT: Decoupled Prompt Tuning
Ji Zhang, Shihan Wu, Lianli Gao +2
This work breaks through the Base-New Tradeoff (BNT)dilemma in prompt tuning, i.e., the better the tuned model generalizes to the base (or target) task, the worse it generalizes to…
MSFlow: Multi-Scale Flow-based Framework for Unsupervised Anomaly Detection
Yixuan Zhou, Xing Xu, Jingkuan Song +2
Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications, where only anomaly-free samples are available for training. Some UAD ap…
CIParsing: Unifying Causality Properties into Multiple Human Parsing
Xiaojia Chen, Xuanhan Wang, Lianli Gao +3
Existing methods of multiple human parsing (MHP) apply statistical models to acquire underlying associations between images and labeled body parts. However, acquired associations o…
Informative Scene Graph Generation via Debiasing
Lianli Gao, Xinyu Lyu, Yuyu Guo +5
Scene graph generation aims to detect visual relationship triplets, (subject, predicate, object). Due to biases in data, current models tend to predict common predicates, e.g. "on"…
MobileVidFactory: Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from Text
Junchen Zhu, Huan Yang, Wenjing Wang +8
Videos for mobile devices become the most popular access to share and acquire information recently. For the convenience of users' creation, in this paper, we present a system, name…
MovieFactory: Automatic Movie Creation from Text using Large Generative Models for Language and Images
Junchen Zhu, Huan Yang, Huiguo He +6
In this paper, we present MovieFactory, a powerful framework to generate cinematic-picture (30721280), film-style (multi-scene), and multi-modality (sounding) movies on the…
CageViT: Convolutional Activation Guided Efficient Vision Transformer
Hao Zheng, Jinbao Wang, Xiantong Zhen +3
Recently, Transformers have emerged as the go-to architecture for both vision and language modeling tasks, but their computational efficiency is limited by the length of the input…
Prototype-based Embedding Network for Scene Graph Generation
Chaofan Zheng, Xinyu Lyu, Lianli Gao +2
Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs. However, due to the diverse visual appearance of numerous p…
Boosting Adversarial Attacks by Leveraging Decision Boundary Information
Boheng Zeng, LianLi Gao, QiLong Zhang +3
Due to the gap between a substitute model and a victim model, the gradient-based noise generated from a substitute model may have low transferability for a victim model since their…
DETA: Denoised Task Adaptation for Few-Shot Learning
Ji Zhang, Lianli Gao, Xu Luo +2
Test-time task adaptation in few-shot learning aims to adapt a pre-trained task-agnostic model for capturing taskspecific knowledge of the test task, rely only on few-labeled suppo…
A Closer Look at Few-shot Classification Again
Xu Luo, Hao Wu, Ji Zhang +3
Few-shot classification consists of a training phase where a model is learned on a relatively large dataset and an adaptation phase where the learned model is adapted to previously…
Hyperbolic Hierarchical Contrastive Hashing
Rukai Wei, Yu Liu, Jingkuan Song +2
Hierarchical semantic structures, naturally existing in real-world datasets, can assist in capturing the latent distribution of data to learn robust hash codes for retrieval system…
Progressive Tree-Structured Prototype Network for End-to-End Image Captioning
Pengpeng Zeng, Jinkuan Zhu, Jingkuan Song +1
Studies of image captioning are shifting towards a trend of a fully end-to-end paradigm by leveraging powerful visual pre-trained models and transformer-based generation architectu…