113 citations · 337 across the 24 of their papers we have counts for
26 papers
StableAnimator: High-Quality Identity-Preserving Human Image Animation
Shuyuan Tu, Zhen Xing, Xintong Han +4
Current diffusion models for human image animation struggle to ensure identity (ID) consistency. This paper presents StableAnimator, the first end-to-end ID-preserving video diffus…
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms
Miaosen Zhang, Yixuan Wei, Zhen Xing +8
Modern vision models are trained on very large noisy datasets. While these models acquire strong capabilities, they may not follow the user's intent to output the desired results i…
AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction
Zhen Xing, Qi Dai, Zejia Weng +2
Text-guided video prediction (TVP) involves predicting the motion of future frames from the initial frame according to an instruction, which has wide applications in virtual realit…
MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion
Shuyuan Tu, Qi Dai, Zihao Zhang +6
Despite impressive advancements in diffusion-based video editing models in altering video attributes, there has been limited exploration into modifying motion information while pre…
VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models
Zhen Xing, Qi Dai, Shuyuan Tu +5
Diffusion models have achieved significant success in image and video generation. This motivates a growing interest in video editing tasks, where videos are edited according to pro…
ARTV: Auto-Regressive Text-to-Video Generation with Diffusion Models
Wenming Weng, Ruoyu Feng, Yanhui Wang +10
We present ARTV, an efficient framework for auto-regressive video generation with diffusion models. Unlike existing methods that generate entire videos in one-s…
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Shuyuan Tu, Qi Dai, Zhi-Qi Cheng +4
Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while…
A Survey on Video Diffusion Models
Zhen Xing, Qijun Feng, Haoran Chen +5
The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision, with the diffusion model playing a crucial role in this achievement. Due to the…
SimDA: Simple Diffusion Adapter for Efficient Video Generation
Zhen Xing, Qi Dai, Han Hu +2
The recent wave of AI-generated content has witnessed the great development and success of Text-to-Image (T2I) technologies. By contrast, Text-to-Video (T2V) still falls short of e…
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
Shuyuan Tu, Qi Dai, Zuxuan Wu +3
Contrastive language-image pretraining (CLIP) has demonstrated remarkable success in various image tasks. However, how to extend CLIP with effective temporal modeling is still an o…
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
Jia Ning, Chen Li, Zheng Zhang +4
Unlike language tasks, where the output space is usually limited to a set of tokens, the output space of visual tasks is more complicated, making it difficult to build a unified vi…
SVFormer: Semi-supervised Video Transformer for Action Recognition
Zhen Xing, Qi Dai, Han Hu +3
Semi-supervised action recognition is a challenging but critical task due to the high cost of video annotations. Existing approaches mainly use convolutional neural networks, yet c…
On Data Scaling in Masked Image Modeling
Zhenda Xie, Zheng Zhang, Yue Cao +4
An important goal of self-supervised learning is to enable model pre-training to benefit from almost unlimited data. However, one method that has recently become popular, namely ma…
Rethinking Spatial Invariance of Convolutional Networks for Object Counting
Zhi-Qi Cheng, Qi Dai, Hong Li +3
Previous work generally believes that improving the spatial invariance of convolutional networks is the key to object counting. However, after verifying several mainstream counting…
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling
Xiaosong Zhang, Yunjie Tian, Wei Huang +4
Recently, masked image modeling (MIM) has offered a new methodology of self-supervised pre-training of vision transformers. A key idea of efficient implementation is to discard the…
Deeper Insights into the Robustness of ViTs towards Common Corruptions
Rui Tian, Zuxuan Wu, Qi Dai +2
With Vision Transformers (ViTs) making great advances in a variety of computer vision tasks, recent literature have proposed various variants of vanilla ViTs to achieve better effi…
SimMIM: A Simple Framework for Masked Image Modeling
Zhenda Xie, Zheng Zhang, Yue Cao +5
This paper presents SimMIM, a simple framework for masked image modeling. We simplify recently proposed related approaches without special designs such as block-wise masking and to…
Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning
Shaobo Min, Qi Dai, Hongtao Xie +3
Cross-modal correlation provides an inherent supervision for video unsupervised representation learning. Existing methods focus on distinguishing different video clips by visual an…
On the Connection between Local Attention and Dynamic Depth-wise Convolution
Qi Han, Zejia Fan, Qi Dai +4
Vision Transformer (ViT) attains state-of-the-art performance in visual recognition, and the variant, Local Vision Transformer, makes further improvements. The major component in L…
Self-Supervised Learning with Swin Transformers
Zhenda Xie, Yutong Lin, Zhuliang Yao +4
We are witnessing a modeling shift from CNN to Transformers in computer vision. In this work, we present a self-supervised learning approach called MoBY, with Vision Transformers a…
Temporal Action Detection with Multi-level Supervision
Baifeng Shi, Qi Dai, Judy Hoffman +3
Training temporal action detection in videos requires large amounts of labeled data, yet such annotation is expensive to collect. Incorporating unlabeled or weakly-labeled data to…
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Baifeng Shi, Dinghuai Zhang, Qi Dai +3
Convolutional Neural Networks (CNNs) are known to rely more on local texture rather than global shape when making decisions. Recent work also indicates a close relationship between…
Weakly-Supervised Action Localization by Generative Attention Modeling
Baifeng Shi, Qi Dai, Yadong Mu +1
Weakly-supervised temporal action localization is a problem of learning an action localization model with only video-level action labeling available. The general framework largely…
Improving the Learning of Multi-column Convolutional Neural Network for Crowd Counting
Zhi-Qi Cheng, Jun-Xiu Li, Qi Dai +3
Tremendous variation in the scale of people/head size is a critical problem for crowd counting. To improve the scale invariance of feature representation, recent works extensively…