7 citations · 42 across the 28 of their papers we have counts for
28 papers
SkyReels-V3 Technique Report
Debang Li, Zhengcong Fei, Tuanhui Li +19
Video generation serves as a cornerstone for building world models, where multimodal contextual inference stands as the defining test of capability. In this end, we present SkyReel…
SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers
Zhengcong Fei, Hao Jiang, Di Qiu +8
The generation and editing of audio-conditioned talking portraits guided by multimodal inputs, including text, images, and videos, remains under explored. In this paper, we present…
SkyReels-V2: Infinite-length Film Generative Model
Guibin Chen, Dixuan Lin, Jiangping Yang +22
Recent advances in video generation have been driven by diffusion models and autoregressive frameworks, yet critical challenges persist in harmonizing prompt adherence, visual qual…
SkyReels-A2: Compose Anything in Video Diffusion Transformers
Zhengcong Fei, Debang Li, Di Qiu +8
This paper presents SkyReels-A2, a controllable video generation framework capable of assembling arbitrary visual elements (e.g., characters, objects, backgrounds) into synthesized…
SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers
Di Qiu, Zhengcong Fei, Rui Wang +5
We present SkyReels-A1, a simple yet effective framework built upon video diffusion Transformer to facilitate portrait image animation. Existing methodologies still encounter issue…
Ingredients: Blending Custom Photos with Video Diffusion Transformers
Zhengcong Fei, Debang Li, Di Qiu +2
This paper presents a powerful framework to customize video creations by incorporating multiple specific identity (ID) photos, with video diffusion Transformers, referred to as Ing…
Video Diffusion Transformers are In-Context Learners
Zhengcong Fei, Di Qiu, Debang Li +2
This paper investigates a solution for enabling in-context capabilities of video diffusion transformers, with minimal tuning required for activation. Specifically, we propose a sim…
FLUX that Plays Music
Zhengcong Fei, Mingyuan Fan, Changqian Yu +1
This paper explores a simple extension of diffusion-based rectified flow Transformers for text-to-music generation, termed as FluxMusic. Generally, along with design in advanced Fl…
Scaling Diffusion Transformers to 16 Billion Parameters
Zhengcong Fei, Mingyuan Fan, Changqian Yu +2
In this paper, we present DiT-MoE, a sparse version of the diffusion Transformer, that is scalable and competitive with dense networks while exhibiting highly optimized inference.…
Dimba: Transformer-Mamba Diffusion Models
Zhengcong Fei, Mingyuan Fan, Changqian Yu +3
This paper unveils Dimba, a new text-to-image diffusion model that employs a distinctive hybrid architecture combining Transformer and Mamba elements. Specifically, Dimba sequentia…
Music Consistency Models
Zhengcong Fei, Mingyuan Fan, Junshi Huang
Consistency models have exhibited remarkable capabilities in facilitating efficient image/video generation, enabling synthesis with minimal sampling steps. It has proven to be adva…
Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models
Zhengcong Fei, Mingyuan Fan, Changqian Yu +2
Transformers have catalyzed advancements in computer vision and natural language processing (NLP) fields. However, substantial computational complexity poses limitations for their…
Scalable Diffusion Models with State Space Backbone
Zhengcong Fei, Mingyuan Fan, Changqian Yu +1
This paper presents a new exploration into a category of diffusion models built upon state space architecture. We endeavor to train diffusion models for image data, wherein the tra…
Tuning-Free Inversion-Enhanced Control for Consistent Image Editing
Xiaoyue Duan, Shuhao Cui, Guoliang Kang +4
Consistent editing of real images is a challenging task, as it requires performing non-rigid edits (e.g., changing postures) to the main objects in the input image without changing…
A-JEPA: Joint-Embedding Predictive Architecture Can Listen
Zhengcong Fei, Mingyuan Fan, Junshi Huang
This paper presents that the masked-modeling principle driving the success of large foundational vision models can be effectively applied to audio by making predictions in a latent…
Prefix-diffusion: A Lightweight Diffusion Model for Diverse Image Captioning
Guisheng Liu, Yi Li, Zhengcong Fei +3
While impressive performance has been achieved in image captioning, the limited diversity of the generated captions and the large parameter scale remain major barriers to the real-…
DiT: Efficient Vision Transformers with Dynamic Token Routing
Yuchen Ma, Zhengcong Fei, Junshi Huang
Recently, the tokens of images share the same static data flow in many dense networks. However, challenges arise from the variance among the objects in images, such as large variat…
Gradient-Free Textual Inversion
Zhengcong Fei, Mingyuan Fan, Junshi Huang
Recent works on personalized text-to-image generation usually learn to bind a special token with specific subjects or styles of a few given images by tuning its embedding through g…
Uncertainty-Aware Image Captioning
Zhengcong Fei, Mingyuan Fan, Li Zhu +3
It is well believed that the higher uncertainty in a word of the caption, the more inter-correlated context information is required to determine it. However, current image captioni…
Meta-Ensemble Parameter Learning
Zhengcong Fei, Shuman Tian, Junshi Huang +2
Ensemble of machine learning models yields improved performance as well as robustness. However, their memory requirements and inference costs can be prohibitively high. Knowledge d…
Progressive Text-to-Image Generation
Zhengcong Fei, Mingyuan Fan, Li Zhu +1
Recently, Vector Quantized AutoRegressive (VQ-AR) models have shown remarkable results in text-to-image synthesis by equally predicting discrete image tokens from the top left to b…
Selecting Stickers in Open-Domain Dialogue through Multitask Learning
Zhexin Zhang, Yeshuang Zhu, Zhengcong Fei +2
With the increasing popularity of online chatting, stickers are becoming important in our online communication. Selecting appropriate stickers in open-domain dialogue requires a co…
Efficient Modeling of Future Context for Image Captioning
Zhengcong Fei, Junshi Huang, Xiaoming Wei +1
Existing approaches to image captioning usually generate the sentence word-by-word from left to right, with the constraint of conditioned on local context including the given image…
DVCFlow: Modeling Information Flow Towards Human-like Video Captioning
Xu Yan, Zhengcong Fei, Shuhui Wang +2
Dense video captioning (DVC) aims to generate multi-sentence descriptions to elucidate the multiple events in the video, which is challenging and demands visual consistency, discou…