89 citations · 398 across the 33 of their papers we have counts for
35 papers
MobileVidFactory: Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from Text
Junchen Zhu, Huan Yang, Wenjing Wang +8
Videos for mobile devices become the most popular access to share and acquire information recently. For the convenience of users' creation, in this paper, we present a system, name…
MovieFactory: Automatic Movie Creation from Text using Large Generative Models for Language and Images
Junchen Zhu, Huan Yang, Huiguo He +6
In this paper, we present MovieFactory, a powerful framework to generate cinematic-picture (30721280), film-style (multi-scene), and multi-modality (sounding) movies on the…
Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learning
Huiguo He, Tianfu Wang, Huan Yang +5
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generatio…
Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution
Yiyang Ma, Huan Yang, Wenhan Yang +2
Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks. However, due to the randomness introduced in the rever…
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
Wenjing Wang, Huan Yang, Zixi Tuo +4
With the explosive popularity of AI-generated content (AIGC), video generation has recently received a lot of attention. Generating videos guided by text instructions poses signifi…
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
Shengming Yin, Chenfei Wu, Huan Yang +13
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation. Most current work generates long videos segment by segment seq…
Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation
Yiyang Ma, Huan Yang, Wenjing Wang +2
Language-guided image generation has achieved great success nowadays by using diffusion models. However, texts can be less detailed to describe highly-specific subjects such as a p…
Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution
Zixi Tuo, Huan Yang, Jianlong Fu +2
Existing real-world video super-resolution (VSR) methods focus on designing a general degradation pipeline for open-domain videos while ignoring data intrinsic characteristics whic…
Improving the detection sensitivity to primordial stochastic gravitational waves with reduced astrophysical foregrounds
Zhen Pan, Huan Yang
One of the primary targets of third-generation (3G) ground-based gravitational wave (GW) detectors is detecting the stochastic GW background (SGWB) from early universe processes. T…
Learning Spatiotemporal Frequency-Transformer for Low-Quality Video Super-Resolution
Zhongwei Qiu, Huan Yang, Jianlong Fu +3
Video Super-Resolution (VSR) aims to restore high-resolution (HR) videos from low-resolution (LR) videos. Existing VSR techniques usually recover HR frames by extracting pertinent…
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang +6
We propose the first joint audio-video generation framework that brings engaging watching and listening experiences simultaneously, towards high-quality realistic videos. To genera…
Fine-Grained Image Style Transfer with Visual Transformers
Jianbo Wang, Huan Yang, Jianlong Fu +2
With the development of the convolutional neural network, image style transfer has drawn increasing attention. However, most existing approaches adopt a global feature transformati…
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song +3
Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks. Previous studies of video-language pretraining mainly focus on shor…
AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation
Yiyang Ma, Huan Yang, Bei Liu +2
AI illustrator aims to automatically design visually appealing images for books to provoke rich thoughts and emotions. To achieve this goal, we propose a framework for translating…
4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement
Chengxu Liu, Huan Yang, Jianlong Fu +1
Image enhancement aims at improving the aesthetic visual quality of photos by retouching the color and tone, and is an essential technology for professional digital photography. Re…
Probing the Delay Time of Supermassive Black Hole Binary Mergers With Gravitational Waves
Yun Fang, Huan Yang
Merging supermassive black hole binaries is expected as a consequence of galaxy mergers, yet the detailed evolution path and underlying merging mechanisms of these binaries are sti…
X-ray view of a merging supermassive black hole binary candidate SDSSJ1430+2303: Results from the first ~200 days of observations
Liming Dou, Ning Jiang, Tinggui Wang +7
Recently we discovered an unprecedented supermassive black hole binary (SMBHB) candidate in the nearby Seyfert galaxy SDSS J1430+2303, which is predicted to merge within three year…
Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution
Zhongwei Qiu, Huan Yang, Jianlong Fu +1
Compressed video super-resolution (VSR) aims to restore high-resolution frames from compressed low-resolution counterparts. Most recent VSR approaches often enhance an input frame…
Repeating Fast Radio Bursts from Neutron Star Binaries: Multi-band and Multi-messenger Opportunities
Zhen Pan, Huan Yang, Kent Yagi
Recent observations indicate that magnetars may commonly reside in merging compact binaries and at least part of fast radio bursts (FRBs) are sourced by magnetar activities. It is…
Language-Guided Face Animation by Recurrent StyleGAN-based Generator
Tiankai Hang, Huan Yang, Bei Liu +3
Recent works on language-guided image manipulation have shown great power of language in providing rich semantics, especially for face images. However, the other natural informatio…
Finding Peas in the Early Universe with JWST
James E. Rhoads, Isak G. B. Wold, Santosh Harish +6
The Early Release Observations (EROs) of JWST beautifully demonstrate the promise of JWST in characterizing the universe at cosmic dawn. We analyze the ERO spectra of three $z \sim…
TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation
Chengxu Liu, Huan Yang, Jianlong Fu +1
Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive frames. State-of-the-art approaches usually adopt a two-step solution, which includ…
Degradation-Guided Meta-Restoration Network for Blind Super-Resolution
Fuzhi Yang, Huan Yang, Yanhong Zeng +2
Blind super-resolution (SR) aims to recover high-quality visual textures from a low-resolution (LR) image, which is usually degraded by down-sampling blur kernels and additive nois…
Learning Trajectory-Aware Transformer for Video Super-Resolution
Chengxu Liu, Huan Yang, Jianlong Fu +1
Video super-resolution (VSR) aims to restore a sequence of high-resolution (HR) frames from their low-resolution (LR) counterparts. Although some progress has been made, there are…