53 citations · 161 across the 15 of their papers we have counts for
16 papers
ViCo: Engaging Video Comment Generation with Human Preference Rewards
Yuchong Sun, Bei Liu, Xu Chen +2
Engaging video comments play an important role in video social media, as they are the carrier of feelings, thoughts, or humor of the audience. Preliminary works have made initial e…
Improving Diversity in Zero-Shot GAN Adaptation with Semantic Variations
Seogkyu Jeon, Bei Liu, Pilhyeon Lee +3
Training deep generative models usually requires a large amount of data. To alleviate the data collection cost, the task of zero-shot GAN adaptation aims to reuse well-trained gene…
Revisiting Latent Space of GAN Inversion for Real Image Editing
Kai Katsumata, Duc Minh Vo, Bei Liu +1
The exploration of the latent space in StyleGANs and GAN inversion exemplify impressive real-world image editing, yet the trade-off between reconstruction quality and editing quali…
Transferring Foundation Models for Generalizable Robotic Manipulation
Jiange Yang, Wenhui Tan, Chuhao Jin +6
Improving the generalization capabilities of general-purpose robotic manipulation agents in the real world has long been a significant challenge. Existing approaches often rely on…
Balancing Reconstruction and Editing Quality of GAN Inversion for Real Image Editing with StyleGAN Prior Latent Space
Kai Katsumata, Duc Minh Vo, Bei Liu +1
The exploration of the latent space in StyleGANs and GAN inversion exemplify impressive real-world image editing, yet the trade-off between reconstruction quality and editing quali…
AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation
Chuhao Jin, Wenhui Tan, Jiange Yang +4
We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks, such as making a smiley face using building blocks. These tasks often invol…
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang +6
We propose the first joint audio-video generation framework that brings engaging watching and listening experiences simultaneously, towards high-quality realistic videos. To genera…
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song +3
Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks. Previous studies of video-language pretraining mainly focus on shor…
AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation
Yiyang Ma, Huan Yang, Bei Liu +2
AI illustrator aims to automatically design visually appealing images for books to provoke rich thoughts and emotions. To achieve this goal, we propose a framework for translating…
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment
Hongwei Xue, Yuchong Sun, Bei Liu +4
The pre-trained image-text models, like CLIP, have demonstrated the strong power of vision-language representation learned from a large scale of web-collected image-text data. In l…
Language-Guided Face Animation by Recurrent StyleGAN-based Generator
Tiankai Hang, Huan Yang, Bei Liu +3
Recent works on language-guided image manipulation have shown great power of language in providing rich semantics, especially for face images. However, the other natural informatio…
Searching the Search Space of Vision Transformer
Minghao Chen, Kan Wu, Bolin Ni +5
Vision Transformer has shown great visual representation power in substantial vision tasks such as recognition and detection, and thus been attracting fast-growing efforts on manua…
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
Hongwei Xue, Tiankai Hang, Yanhong Zeng +5
We study joint video and language (VL) pre-training to enable cross-modality learning and benefit plentiful downstream VL tasks. Existing works either extract low-quality video fea…
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
Yupan Huang, Bei Liu, Jianlong Fu +1
A creative image-and-text generative AI system mimics humans' extraordinary abilities to provide users with diverse and comprehensive caption suggestions, as well as rich image cre…
Unifying Multimodal Transformer for Bi-directional Image and Text Generation
Yupan Huang, Hongwei Xue, Bei Liu +1
We study the joint learning of image-to-text and text-to-image generations, which are naturally bi-directional tasks. Typical existing works design two separate task-specific model…
Learning Fine-Grained Motion Embedding for Landscape Animation
Hongwei Xue, Bei Liu, Huan Yang +3
In this paper we focus on landscape animation, which aims to generate time-lapse videos from a single landscape image. Motion is crucial for landscape animation as it determines ho…