25 citations · 27 across the 18 of their papers we have counts for
23 papers
Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors
Henry Che, Xuanchi Ren, Sanja Fidler +3
We present LidarVerse, a real-time unified LiDAR world model for sensor simulation across tasks and sensors. Existing LiDAR simulators largely follow one of two complementary parad…
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation
Cong Wei, Xuanchi Ren, Bryan Chu +8
Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with…
Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
Hmrishav Bandyopadhyay, Xuanchi Ren, Zijian Huang +7
Interactive autoregressive video generation demands both low-latency rollouts and precise online control. Few-step distillation accelerates generation by reducing denoising steps,…
NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
NVIDIA, :, Aarti Basant +33
As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving pol…
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
APE: Agentic Prompt Enhancer for Image Generation and Editing
Zijian Huang, Jay Zhangjie Wu, Zian Wang +5
Natural language has become a powerful interface for image generation and editing, yet text-guided visual systems remain highly sensitive to prompt formulation. Semantically simila…
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players
Fangfu Liu, Kai He, Tianchang Shen +7
World models for interactive video generation have largely focused on single-agent settings, where future observations are generated from a single control signal. However, many gen…
PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion
Yifan Lu, Qi Wu, Jay Zhangjie Wu +4
Most practical high-resolution text-to-image systems, including latent diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the gen…
Lyra 2.0: Explorable Generative 3D Worlds
Tianchang Shen, Sherwin Bahmani, Kai He +12
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via f…
MoRight: Motion Control Done Right
Shaowei Liu, Xuanchi Ren, Tianchang Shen +5
Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangl…
ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
Riccardo de Lutio, Tobias Fischer, Yen-Yu Chang +7
Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Methods that leve…
Depth Completion as Parameter-Efficient Test-Time Adaptation
Bingxin Ke, Qunjie Zhou, Jiahui Huang +5
We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse geometric cues. Unli…
World Simulation with Video Foundation Models for Physical AI
NVIDIA, :, Arslan Ali +87
We introduce [Cosmos-Predict2.5], the latest generation of the Cosmos World Foundation Models for Physical AI. Built on a flow-based architecture, [Cosmos-Predict2.5] unifies Text2…
ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation
Jay Zhangjie Wu, Xuanchi Ren, Tianchang Shen +11
Recent advances in large generative models have greatly enhanced both image editing and in-context image generation, yet a critical gap remains in ensuring physical consistency, wh…
Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Sherwin Bahmani, Tianchang Shen, Jiawei Ren +10
The ability to generate virtual environments is crucial for applications ranging from gaming to physical AI domains such as robotics, autonomous driving, and industrial AI. Current…
ViPE: Video Pose Engine for 3D Geometric Perception
Jiahui Huang, Qunjie Zhou, Hesam Rabeti +12
Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring…
Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
Xuanchi Ren, Yifan Lu, Tianshi Cao +13
Collecting and annotating real-world data for safety-critical physical AI systems, such as Autonomous Vehicle (AV), is time-consuming and costly. It is especially challenging to ca…
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
NVIDIA, :, Hassan Abu Alhaija +38
We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmen…
Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki +6
Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D reconstruction and novel-view synthesis task. However, achieving photorealistic rendering from extreme novel…
InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models
Yifan Lu, Xuanchi Ren, Jiawei Yang +8
We present InfiniCube, a scalable method for generating unbounded dynamic 3D driving scenes with high fidelity and controllability. Previous methods for scene generation either suf…
SCube: Instant Large-Scale Scene Reconstruction using VoxSplats
Xuanchi Ren, Yifan Lu, Hanxue Liang +6
We present SCube, a novel method for reconstructing large-scale 3D scenes (geometry, appearance, and semantics) from a sparse set of posed images. Our method encodes reconstructed…
fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence
Francis Williams, Jiahui Huang, Jonathan Swartz +9
We present fVDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. fVDB provides a complete set of differentiable primitives to build deep learning architec…
XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies
Xuanchi Ren, Jiahui Huang, Xiaohui Zeng +3
We present XCube (abbreviated as ), a novel generative model for high-resolution sparse 3D voxel grids with arbitrary attributes. Our model can generate millions of…