283 citations · 1.4k across the 72 of their papers we have counts for
13 papers · 1 filter
ViCo: Engaging Video Comment Generation with Human Preference Rewards
Yuchong Sun, Bei Liu, Xu Chen +2
Engaging video comments play an important role in video social media, as they are the carrier of feelings, thoughts, or humor of the audience. Preliminary works have made initial e…
Improving Diversity in Zero-Shot GAN Adaptation with Semantic Variations
Seogkyu Jeon, Bei Liu, Pilhyeon Lee +3
Training deep generative models usually requires a large amount of data. To alleviate the data collection cost, the task of zero-shot GAN adaptation aims to reuse well-trained gene…
SINC: Self-Supervised In-Context Learning for Vision-Language Tasks
Yi-Syuan Chen, Yun-Zhu Song, Cheng Yu Yeo +3
Large Pre-trained Transformers exhibit an intriguing capacity for in-context learning. Without gradient updates, these models can rapidly construct new predictors from demonstratio…
MobileVidFactory: Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from Text
Junchen Zhu, Huan Yang, Wenjing Wang +8
Videos for mobile devices become the most popular access to share and acquire information recently. For the convenience of users' creation, in this paper, we present a system, name…
MovieFactory: Automatic Movie Creation from Text using Large Generative Models for Language and Images
Junchen Zhu, Huan Yang, Huiguo He +6
In this paper, we present MovieFactory, a powerful framework to generate cinematic-picture (30721280), film-style (multi-scene), and multi-modality (sounding) movies on the…
Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learning
Huiguo He, Tianfu Wang, Huan Yang +5
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generatio…
Transferring Foundation Models for Generalizable Robotic Manipulation
Jiange Yang, Wenhui Tan, Chuhao Jin +6
Improving the generalization capabilities of general-purpose robotic manipulation agents in the real world has long been a significant challenge. Existing approaches often rely on…
AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation
Chuhao Jin, Wenhui Tan, Jiange Yang +4
We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks, such as making a smiley face using building blocks. These tasks often invol…
Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution
Yiyang Ma, Huan Yang, Wenhan Yang +2
Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks. However, due to the randomness introduced in the rever…
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
Wenjing Wang, Huan Yang, Zixi Tuo +4
With the explosive popularity of AI-generated content (AIGC), video generation has recently received a lot of attention. Generating videos guided by text instructions poses signifi…
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
Shengming Yin, Chenfei Wu, Huan Yang +13
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation. Most current work generates long videos segment by segment seq…
Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation
Yiyang Ma, Huan Yang, Wenjing Wang +2
Language-guided image generation has achieved great success nowadays by using diffusion models. However, texts can be less detailed to describe highly-specific subjects such as a p…
Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution
Zixi Tuo, Huan Yang, Jianlong Fu +2
Existing real-world video super-resolution (VSR) methods focus on designing a general degradation pipeline for open-domain videos while ignoring data intrinsic characteristics whic…