26 citations · 47 across the 13 of their papers we have counts for
14 papers
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
E-4DGS: High-Fidelity Dynamic Reconstruction from the Multi-view Event Cameras
Chaoran Feng, Zhenyu Tang, Wangbo Yu +5
Novel view synthesis and 4D reconstruction techniques predominantly rely on RGB cameras, thereby inheriting inherent limitations such as the dependence on adequate lighting, suscep…
UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation
Bin Lin, Zongjian Li, Xinhua Cheng +9
Although existing unified models achieve strong performance in vision-language understanding and text-to-image generation, they remain limited in addressing image perception and ma…
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Mingzhe Zheng, Yongqi Xu, Haojian Huang +8
Current video generation models excel at short clips but fail to produce cohesive multi-shot narratives due to disjointed visual dynamics and fractured storylines. Existing solutio…
SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Video
Chengshu Zhao, Yunyang Ge, Xinhua Cheng +6
Video body-swapping aims to replace the body in an existing video with a new body from arbitrary sources, which has garnered more attention in recent years. Existing methods treat…
Next Patch Prediction for Autoregressive Visual Generation
Yatian Pang, Peng Jin, Shuo Yang +8
Autoregressive models, built based on the Next Token Prediction (NTP) paradigm, show great potential in developing a unified framework that integrates both language and vision task…
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Mingzhe Zheng, Yongqi Xu, Haojian Huang +8
Current video generation models excel at short clips but fail to produce cohesive multi-shot narratives due to disjointed visual dynamics and fractured storylines. Existing solutio…
DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses
Yatian Pang, Bin Zhu, Bin Lin +5
In this work, we present DreamDance, a novel method for animating human images using only skeleton pose sequences as conditional inputs. Existing approaches struggle with generatin…
Open-Sora Plan: Open-Source Large Video Generation Model
Bin Lin, Yunyang Ge, Xinhua Cheng +21
We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on vari…
Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle
Zhenyu Tang, Junwu Zhang, Xinhua Cheng +5
Recent 3D large reconstruction models typically employ a two-stage process, including first generate multi-view images by a multi-view diffusion model, and then utilize a feed-forw…
Envision3D: One Image to 3D with Anchor Views Interpolation
Yatian Pang, Tanghui Jia, Yujun Shi +6
We present Envision3D, a novel method for efficiently generating high-quality 3D content from a single image. Recent methods that extract 3D content from multi-view images generate…
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting
Junwu Zhang, Zhenyu Tang, Yatian Pang +5
Recent one image to 3D generation methods commonly adopt Score Distillation Sampling (SDS). Despite the impressive results, there are multiple deficiencies including multi-view inc…
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
Bin Zhu, Bin Lin, Munan Ning +11
The video-language (VL) pretraining has achieved remarkable improvement in multiple downstream tasks. However, the current VL pretraining framework is hard to extend to multiple mo…
Masked Autoencoders for Point Cloud Self-supervised Learning
Yatian Pang, Wenxiao Wang, Francis E. H. Tay +3
As a promising scheme of self-supervised learning, masked autoencoding has significantly advanced natural language processing and computer vision. Inspired by this, we propose a ne…