283 citations · 1.4k across the 72 of their papers we have counts for
18 papers · 1 filter
Learning Spatiotemporal Frequency-Transformer for Low-Quality Video Super-Resolution
Zhongwei Qiu, Huan Yang, Jianlong Fu +3
Video Super-Resolution (VSR) aims to restore high-resolution (HR) videos from low-resolution (LR) videos. Existing VSR techniques usually recover HR frames by extracting pertinent…
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang +6
We propose the first joint audio-video generation framework that brings engaging watching and listening experiences simultaneously, towards high-quality realistic videos. To genera…
Weakly-supervised Pre-training for 3D Human Pose Estimation via Perspective Knowledge
Zhongwei Qiu, Kai Qiu, Jianlong Fu +1
Modern deep learning-based 3D pose estimation approaches require plenty of 3D pose annotations. However, existing 3D datasets lack diversity, which limits the performance of curren…
Fine-Grained Image Style Transfer with Visual Transformers
Jianbo Wang, Huan Yang, Jianlong Fu +2
With the development of the convolutional neural network, image style transfer has drawn increasing attention. However, most existing approaches adopt a global feature transformati…
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song +3
Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks. Previous studies of video-language pretraining mainly focus on shor…
AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation
Yiyang Ma, Huan Yang, Bei Liu +2
AI illustrator aims to automatically design visually appealing images for books to provoke rich thoughts and emotions. To achieve this goal, we propose a framework for translating…
4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement
Chengxu Liu, Huan Yang, Jianlong Fu +1
Image enhancement aims at improving the aesthetic visual quality of photos by retouching the color and tone, and is an essential technology for professional digital photography. Re…
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment
Hongwei Xue, Yuchong Sun, Bei Liu +4
The pre-trained image-text models, like CLIP, have demonstrated the strong power of vision-language representation learned from a large scale of web-collected image-text data. In l…
Exploring Anchor-based Detection for Ego4D Natural Language Query
Sipeng Zheng, Qi Zhang, Bei Liu +2
In this paper we provide the technique report of Ego4D natural language query challenge in CVPR 2022. Natural language query task is challenging due to the requirement of comprehen…
GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training
Jaeseok Byun, Taebaek Hwang, Jianlong Fu +1
Most of the currently existing vision and language pre-training (VLP) methods have mainly focused on how to extract and align vision and text features. In contrast to the mainstrea…
Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution
Zhongwei Qiu, Huan Yang, Jianlong Fu +1
Compressed video super-resolution (VSR) aims to restore high-resolution frames from compressed low-resolution counterparts. Most recent VSR approaches often enhance an input frame…
Expanding Language-Image Pretrained Models for General Video Recognition
Bolin Ni, Houwen Peng, Minghao Chen +5
Contrastive language-image pretraining has shown great success in learning visual-textual joint representation from web-scale data, demonstrating remarkable "zero-shot" generalizat…
Language-Guided Face Animation by Recurrent StyleGAN-based Generator
Tiankai Hang, Huan Yang, Bei Liu +3
Recent works on language-guided image manipulation have shown great power of language in providing rich semantics, especially for face images. However, the other natural informatio…
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Kan Wu, Jinnian Zhang, Houwen Peng +4
Vision transformer (ViT) recently has drawn great attention in computer vision due to its remarkable model capability. However, most prevailing ViT models suffer from huge number o…
TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation
Chengxu Liu, Huan Yang, Jianlong Fu +1
Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive frames. State-of-the-art approaches usually adopt a two-step solution, which includ…
Degradation-Guided Meta-Restoration Network for Blind Super-Resolution
Fuzhi Yang, Huan Yang, Yanhong Zeng +2
Blind super-resolution (SR) aims to recover high-quality visual textures from a low-resolution (LR) image, which is usually degraded by down-sampling blur kernels and additive nois…
Learning Trajectory-Aware Transformer for Video Super-Resolution
Chengxu Liu, Huan Yang, Jianlong Fu +1
Video super-resolution (VSR) aims to restore a sequence of high-resolution (HR) frames from their low-resolution (LR) counterparts. Although some progress has been made, there are…
MiniViT: Compressing Vision Transformers with Weight Multiplexing
Jinnian Zhang, Houwen Peng, Kan Wu +4
Vision Transformer (ViT) models have recently drawn much attention in computer vision due to their high model capability. However, ViT models suffer from huge number of parameters,…