2 citations · 4 across the 14 of their papers we have counts for
14 papers
Playing with Transformer at 30+ FPS via Next-Frame Diffusion
Xinle Cheng, Tianyu He, Jiayi Xu +3
Autoregressive video models offer distinct advantages over bidirectional diffusion models in creating interactive video content and supporting streaming applications with arbitrary…
Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling
Haoyu Wu, Diankun Wu, Tianyu He +4
Videos inherently represent 2D projections of a dynamic 3D world. However, our analysis suggests that video diffusion models trained solely on raw video data often fail to capture…
MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft
Junliang Guo, Yang Ye, Tianyu He +4
World modeling is a crucial task for enabling intelligent agents to effectively interact with humans and operate in dynamic environments. In this work, we propose MineWorld, a real…
Fast Autoregressive Video Generation with Diagonal Decoding
Yang Ye, Junliang Guo, Haoyu Wu +5
Autoregressive Transformer models have demonstrated impressive performance in video generation, but their sequential token-by-token decoding process poses a major bottleneck, parti…
AR4D: Autoregressive 4D Generation from Monocular Videos
Hanxin Zhu, Tianyu He, Xiqian Yu +3
Recent advancements in generative models have ignited substantial interest in dynamic 3D content creation (\ie, 4D generation). Existing approaches primarily rely on Score Distilla…
VidTok: A Versatile and Open-Source Video Tokenizer
Anni Tang, Tianyu He, Junliang Guo +3
Encoding video content into compact latent tokens has become a fundamental step in video generation and understanding, driven by the need to address the inherent redundancy in pixe…
VidTwin: Video VAE with Decoupled Structure and Dynamics
Yuchi Wang, Junliang Guo, Xinyi Xie +3
Recent advancements in video autoencoders (Video AEs) have significantly improved the quality and efficiency of video generation. In this paper, we propose a novel and compact vide…
Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning
Bei Li, Tong Zheng, Rui Wang +8
Residual networks, as discrete approximations of Ordinary Differential Equations (ODEs), have inspired significant advancements in neural network design, including multistep method…
IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI
Xiaoyu Chen, Junliang Guo, Tianyu He +5
We introduce Image-GOal Representations (IGOR), aiming to learn a unified, semantically consistent action space across human and various robots. Through this unified latent action…
Compositional 3D-aware Video Generation with LLM Director
Hanxin Zhu, Tianyu He, Anni Tang +3
Significant progress has been made in text-to-video generation through the use of powerful generative models and large-scale internet data. However, substantial challenges remain i…
Make Your Actor Talk: Generalizable and High-Fidelity Lip Sync with Motion and Appearance Disentanglement
Runyi Yu, Tianyu He, Ailing Zhang +6
We aim to edit the lip movements in talking video according to the given speech while preserving the personal identity and visual details. The task can be decomposed into two sub-p…
InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generation
Yuchi Wang, Junliang Guo, Jianhong Bai +5
Recent talking avatar generation models have made strides in achieving realistic and accurate lip synchronization with the audio, but often fall short in controlling and conveying…
Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training
Qingyan Guo, Rui Wang, Junliang Guo +3
While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typ…
UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing
Jianhong Bai, Tianyu He, Yuchi Wang +4
Recent advances in text-guided video editing have showcased promising results in appearance editing (e.g., stylization). However, video motion editing in the temporal dimension (e.…