285 citations · 499 across the 33 of their papers we have counts for
33 papers
HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generation
Xiang Wang, Zhifei Zhang, He Zhang +11
Recent unified models integrate understanding experts (e.g., LLMs) with generative experts (e.g., diffusion models), achieving strong multimodal performance. However, recent advanc…
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
Zhiwu Qing, Shiwei Zhang, Ziyuan Huang +4
Recently, large-scale pre-trained language-image models like CLIP have shown extraordinary capabilities for understanding spatial contents, but naively transferring such models to…
Towards Real-World Visual Tracking with Temporal Contexts
Ziang Cao, Ziyuan Huang, Liang Pan +3
Visual tracking has made significant improvements in the past few decades. Most existing state-of-the-art trackers 1) merely aim for performance in ideal conditions while overlooki…
Temporally-Adaptive Models for Efficient Video Understanding
Ziyuan Huang, Shiwei Zhang, Liang Pan +4
Spatial convolutions are extensively used in numerous deep video models. It fundamentally assumes spatio-temporal invariance, i.e., using shared weights for every location in diffe…
VideoComposer: Compositional Video Synthesis with Motion Controllability
Xiang Wang, Hangjie Yuan, Shiwei Zhang +6
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis. However, achieving controllable vide…
MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Zhiwu Qing +4
Current state-of-the-art approaches for few-shot action recognition achieve promising performance by conducting frame-level matching on learned visual features. However, they gener…
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
Jun Cen, Shiwei Zhang, Xiang Wang +4
Open-set action recognition is to reject unknown human action cases which are out of the distribution of the training set. Existing methods mainly focus on learning better uncertai…
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Jun Cen +4
Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explore…
HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Zhiwu Qing +4
Recent attempts mainly focus on learning deep representations for each video individually under the episodic meta-learning regime and then performing temporal alignment to match qu…
Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
Yixuan Pei, Zhiwu Qing, Jun Cen +6
Recent incremental learning for action recognition usually stores representative videos to mitigate catastrophic forgetting. However, only a few bulky videos can be stored due to t…
MAR: Masked Autoencoders for Efficient Action Recognition
Zhiwu Qing, Shiwei Zhang, Ziyuan Huang +5
Standard approaches for video recognition usually operate on the full input videos, which is inefficient due to the widely present spatio-temporal redundancy in videos. Recent prog…
Open-world Semantic Segmentation for LIDAR Point Clouds
Jun Cen, Peng Yun, Shiwei Zhang +5
Current methods for LIDAR semantic segmentation are not robust enough for real-world applications, e.g., autonomous driving, since it is closed-set and static. The closed-set assum…
Context-aware Proposal Network for Temporal Action Detection
Xiang Wang, Huaxin Zhang, Shiwei Zhang +3
This technical report presents our first place winning solution for temporal action detection task in CVPR-2022 AcitivityNet Challenge. The task aims to localize temporal boundarie…
Hybrid Relation Guided Set Matching for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Zhiwu Qing +5
Current few-shot action recognition methods reach impressive performance by learning discriminative features for each video via episodic training and designing various temporal ali…
Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency
Zhiwu Qing, Shiwei Zhang, Ziyuan Huang +6
Natural videos provide rich visual contents for self-supervised learning. Yet most existing approaches for learning spatio-temporal representations rely on manually trimmed videos,…
TCTrack: Temporal Contexts for Aerial Tracking
Ziang Cao, Ziyuan Huang, Liang Pan +3
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers. In this work, we present TCTrack, a comprehensive framework to fully explo…
TAda! Temporally-Adaptive Convolutions for Video Understanding
Ziyuan Huang, Shiwei Zhang, Liang Pan +4
Spatial convolutions are widely used in numerous deep video models. It fundamentally assumes spatio-temporal invariance, i.e., using shared weights for every location in different…
Support-Set Based Cross-Supervision for Video Grounding
Xinpeng Ding, Nannan Wang, Shiwei Zhang +5
Current approaches for video grounding propose kinds of complex architectures to capture the video-text relations, and have achieved impressive improvements. However, it is hard to…
ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
Zhiwu Qing, Ziyuan Huang, Shiwei Zhang +5
The central idea of contrastive learning is to discriminate between different instances and force different views from the same instance to share the same representation. To avoid…
Exploring Stronger Feature for Temporal Action Localization
Zhiwu Qing, Xiang Wang, Ziyuan Huang +6
Temporal action localization aims to localize starting and ending time with action category. Limited by GPU memory, mainstream methods pre-extract features for each video. Therefor…
OadTR: Online Action Detection with Transformers
Xiang Wang, Shiwei Zhang, Zhiwu Qing +4
Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure. However, RNN suffers from non-parallelism…
Proposal Relation Network for Temporal Action Detection
Xiang Wang, Zhiwu Qing, Ziyuan Huang +6
This technical report presents our solution for temporal action detection task in AcitivityNet Challenge 2021. The purpose of this task is to locate and identify actions of interes…
Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
Xiang Wang, Zhiwu Qing, Ziyuan Huang +6
Weakly-Supervised Temporal Action Localization (WS-TAL) task aims to recognize and localize temporal starts and ends of action instances in an untrimmed video with only video-level…
Relation Modeling in Spatio-Temporal Action Localization
Yutong Feng, Jianwen Jiang, Ziyuan Huang +5
This paper presents our solution to the AVA-Kinetics Crossover Challenge of ActivityNet workshop at CVPR 2021. Our solution utilizes multiple types of relation modeling methods for…