11 citations · 40 across the 36 of their papers we have counts for
42 papers
Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors
Henry Che, Xuanchi Ren, Sanja Fidler +3
We present LidarVerse, a real-time unified LiDAR world model for sensor simulation across tasks and sensors. Existing LiDAR simulators largely follow one of two complementary parad…
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation
Cong Wei, Xuanchi Ren, Bryan Chu +8
Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with…
Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
Hmrishav Bandyopadhyay, Xuanchi Ren, Zijian Huang +7
Interactive autoregressive video generation demands both low-latency rollouts and precise online control. Few-step distillation accelerates generation by reducing denoising steps,…
NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
NVIDIA, :, Aarti Basant +33
As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving pol…
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
A Scalable PyTorch Abstraction for Multi-GPU Gaussian Splatting
Matthew Cong, Francis Williams, Jonathan Swartz +3
Gaussian splatting methods have become increasingly popular for neural reconstruction of the real world. However, they are often limited in scale and resolution due to compute and…
TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions
Or Perel, Hassan Abu Alhaija, Zian Wang +4
We introduce TRON, a rendering framework that combines 3D Gaussian ray tracing with neural rendering to enable realistic and controllable rendering of real-world 3D scenes under no…
GRAIL: Generating Humanoid Loco-Manipulation from World Models
Tianyi Xie, Haotian Zhang, Jinhyung Park +17
Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture a…
APE: Agentic Prompt Enhancer for Image Generation and Editing
Zijian Huang, Jay Zhangjie Wu, Zian Wang +5
Natural language has become a powerful interface for image generation and editing, yet text-guided visual systems remain highly sensitive to prompt formulation. Semantically simila…
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players
Fangfu Liu, Kai He, Tianchang Shen +7
World models for interactive video generation have largely focused on single-agent settings, where future observations are generated from a single control signal. However, many gen…
PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion
Yifan Lu, Qi Wu, Jay Zhangjie Wu +4
Most practical high-resolution text-to-image systems, including latent diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the gen…
Lyra 2.0: Explorable Generative 3D Worlds
Tianchang Shen, Sherwin Bahmani, Kai He +12
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via f…
MoRight: Motion Control Done Right
Shaowei Liu, Xuanchi Ren, Tianchang Shen +5
Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangl…
Kimodo: Scaling Controllable Human Motion Generation
Davis Rempe, Mathis Petrovich, Ye Yuan +21
High-quality human motion data is becoming increasingly important for applications in robotics, simulation, and entertainment. Recent generative models offer a potential data sourc…
DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer
Yuxuan Zhang, Katarína Tóthová, Zian Wang +7
Simulation is essential to the development and evaluation of autonomous robots such as self-driving vehicles. Neural reconstruction is emerging as a promising solution as it enable…
Motion Attribution for Video Generation
Xindi Wu, Despoina Paschalidou, Jun Gao +5
Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a m…
Test-time scaling of diffusions with flow maps
Amirmojtaba Sabour, Michael S. Albergo, Carles Domingo-Enrich +4
A common recipe to improve diffusion models at test-time so that samples score highly against a user-specified reward is to introduce the gradient of the reward into the dynamics o…
World Simulation with Video Foundation Models for Physical AI
NVIDIA, :, Arslan Ali +87
We introduce [Cosmos-Predict2.5], the latest generation of the Cosmos World Foundation Models for Physical AI. Built on a flow-based architecture, [Cosmos-Predict2.5] unifies Text2…
ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation
Jay Zhangjie Wu, Xuanchi Ren, Tianchang Shen +11
Recent advances in large generative models have greatly enhanced both image editing and in-context image generation, yet a critical gap remains in ensuring physical consistency, wh…
Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Sherwin Bahmani, Tianchang Shen, Jiawei Ren +10
The ability to generate virtual environments is crucial for applications ranging from gaming to physical AI domains such as robotics, autonomous driving, and industrial AI. Current…
ViPE: Video Pose Engine for 3D Geometric Perception
Jiahui Huang, Qunjie Zhou, Hesam Rabeti +12
Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring…
Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
Xuanchi Ren, Yifan Lu, Tianshi Cao +13
Collecting and annotating real-world data for safety-critical physical AI systems, such as Autonomous Vehicle (AV), is time-consuming and costly. It is especially challenging to ca…
Align Your Flow: Scaling Continuous-Time Flow Map Distillation
Amirmojtaba Sabour, Sanja Fidler, Karsten Kreis
Diffusion- and flow-based models have emerged as state-of-the-art generative modeling approaches, but they require many sampling steps. Consistency models can distill these models…
PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond
Minghua Liu, Mikaela Angelina Uy, Donglai Xiang +4
We propose PartField, a feedforward approach for learning part-based 3D features, which captures the general concept of parts and their hierarchy without relying on predefined temp…