285 citations · 966 across the 40 of their papers we have counts for
26 papers · 1 filter
VDT: General-purpose Video Diffusion Transformers via Mask Modeling
Haoyu Lu, Guoxing Yang, Nanyi Fei +4
This work introduces Video Diffusion Transformer (VDT), which pioneers the use of transformers in diffusion-based video generation. It features transformer blocks with modularized…
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
Haoyu Lu, Yuqi Huo, Guoxing Yang +4
Large-scale vision-language pre-trained models have shown promising transferability to various downstream tasks. As the size of these foundation models and the number of downstream…
LGDN: Language-Guided Denoising Network for Video-Language Modeling
Haoyu Lu, Mingyu Ding, Nanyi Fei +2
Video-language modeling has attracted much attention with the rapid growth of web videos. Most existing methods assume that the video frames and text description are semantically c…
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Haoyu Lu, Nanyi Fei, Yuqi Huo +3
Large-scale single-stream pre-training has shown dramatic performance in image-text retrieval. Regrettably, it faces low inference efficiency due to heavy attention layers. Recentl…
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
Mingyu Ding, Xiaochen Lian, Linjie Yang +4
High-resolution representations (HR) are essential for dense prediction tasks such as segmentation, detection, and pose estimation. Learning HR representations is typically ignored…
Learning Versatile Neural Architectures by Propagating Network Codes
Mingyu Ding, Yuqi Huo, Haoyu Lu +5
This work explores how to design a single neural network capable of adapting to multiple heterogeneous vision tasks, such as image segmentation, 3D detection, and video recognition…
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
Yuqi Huo, Manli Zhang, Guangzhen Liu +32
Multi-modal pre-training models have been intensively explored to bridge vision and language in recent years. However, most of them explicitly model the cross-modal interaction bet…
Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning
Yizhao Gao, Nanyi Fei, Guangzhen Liu +3
Most recent few-shot learning (FSL) methods are based on meta-learning with episodic training. In each meta-training episode, a discriminative feature embedding and/or classifier a…
Counterfactual VQA: A Cause-Effect Look at Language Bias
Yulei Niu, Kaihua Tang, Hanwang Zhang +3
VQA models may tend to rely on language bias as a shortcut and thus fail to sufficiently learn the multi-modal knowledge from both vision and language. Recent debiasing methods pro…
Domain-Adaptive Few-Shot Learning
An Zhao, Mingyu Ding, Zhiwu Lu +5
Existing few-shot learning (FSL) methods make the implicit assumption that the few target class samples are from the same domain as the source class samples. However, in practice t…
Learning Depth-Guided Convolutions for Monocular 3D Object Detection
Mingyu Ding, Yuqi Huo, Hongwei Yi +4
3D object detection from a single image without LiDAR is a challenging task due to the lack of accurate depth information. Conventional 2D convolutions are unsuitable for this task…
Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow
Mingyu Ding, Zhe Wang, Bolei Zhou +3
A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised…
Mobile Video Action Recognition
Yuqi Huo, Xiaoli Xu, Yao Lu +3
Video action recognition, which is topical in computer vision and video analysis, aims to allocate a short video clip to a pre-defined category such as brushing hair or climbing st…
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
Yulei Niu, Hanwang Zhang, Zhiwu Lu +1
We focus on grounding (i.e., localizing or linking) referring expressions in images, e.g., ``largest elephant standing behind baby elephant''. This is a general yet challenging vis…
Face-Focused Cross-Stream Network for Deception Detection in Videos
Mingyu Ding, An Zhao, Zhiwu Lu +2
Automated deception detection (ADD) from real-life videos is a challenging task. It specifically needs to address two problems: (1) Both face and body contain useful cues regarding…
Zero-Shot Learning with Sparse Attribute Propagation
Nanyi Fei, Jiechao Guan, Zhiwu Lu +2
Zero-shot learning (ZSL) aims to recognize a set of unseen classes without any training images. The standard approach to ZSL requires a set of training images annotated with seen c…
Recursive Visual Attention in Visual Dialog
Yulei Niu, Hanwang Zhang, Manli Zhang +3
Visual dialog is a challenging vision-language task, which requires the agent to answer multi-round questions about an image. It typically needs to address two major problems: (1)…
Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
Zhiwu Lu, Jiechao Guan, Aoxue Li +3
Zero-shot learning (ZSL) is made possible by learning a projection function between a feature space and a semantic space (e.g.,~an attribute space). Key to ZSL is thus to learn a p…
Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning
Aoxue Li, Zhiwu Lu, Jiechao Guan +3
Zero-shot learning (ZSL) aims to transfer knowledge from seen classes to unseen ones so that the latter can be recognised without any training samples. This is made possible by lea…
Domain-Invariant Projection Learning for Zero-Shot Recognition
An Zhao, Mingyu Ding, Jiechao Guan +3
Zero-shot learning (ZSL) aims to recognize unseen object classes without any training samples, which can be regarded as a form of transfer learning from seen classes to unseen ones…
Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation
Yulei Niu, Zhiwu Lu, Ji-Rong Wen +2
Image annotation aims to annotate a given image with a variable number of class labels corresponding to diverse visual concepts. In this paper, we address two main issues in large-…
Zero-Shot Fine-Grained Classification by Deep Feature Learning with Semantics
Aoxue Li, Zhiwu Lu, Liwei Wang +3
Fine-grained image classification, which aims to distinguish images with subtle distinctions, is a challenging task due to two main issues: lack of sufficient training data for eve…
Pairwise Constraint Propagation: A Survey
Zhenyong Fu, Zhiwu Lu
As one of the most important types of (weaker) supervised information in machine learning and pattern recognition, pairwise constraint, which specifies whether a pair of data point…
Image classification by visual bag-of-words refinement and reduction
Zhiwu Lu, Liwei Wang, Ji-Rong Wen
This paper presents a new framework for visual bag-of-words (BOW) refinement and reduction to overcome the drawbacks associated with the visual BOW model which has been widely used…