5 papers
Cosmos World Foundation Model Platform for Physical AI
NVIDIA, :, Niket Agarwal +76
Physical AI needs to be trained digitally first. It needs a digital twin of itself, the policy model, and a digital twin of the world, the world model. In this paper, we present th…
Training Video Foundation Models with NVIDIA NeMo
Zeeshan Patel, Ethan He, Parth Mannan +26
Video Foundation Models (VFMs) have recently been used to simulate the real world to train physical AI systems and develop creative visual experiences. However, there are significa…
Exploring Diffusion and Flow Matching Under Generator Matching
Zeeshan Patel, James DeLoye, Lance Mathias
In this paper, we present a comprehensive theoretical comparison of diffusion and flow matching under the Generator Matching framework. Despite their apparent differences, both dif…
Scaling Properties of Diffusion Models for Perceptual Tasks
Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran +1
In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as…
SWAG: Storytelling With Action Guidance
Zeeshan Patel, Karim El-Refai, Jonathan Pei +1
Automated long-form story generation typically employs long-context large language models (LLMs) for one-shot creation, which can produce cohesive but not necessarily engaging cont…