5 papers
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
Safe Vision-Language Models via Unsafe Weights Manipulation
Moreno D'IncÃ, Elia Peruzzo, Xingqian Xu +3
Vision-language models (VLMs) often inherit the biases and unsafe associations present within their large-scale training dataset. While recent approaches mitigate unsafe behaviors,…
IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance
Jiayi Guo, Chuanhao Yan, Xingqian Xu +4
Ensuring precise multimodal alignment between diffusion-generated images and input prompts has been a long-standing challenge. Earlier works finetune diffusion weight using high-qu…
Efficient Image Generation with Variadic Attention Heads
Steven Walton, Ali Hassani, Xingqian Xu +2
While the integration of transformers in vision models have yielded significant improvements on vision tasks they still require significant amounts of computation for both training…
RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism
Elia Peruzzo, Dejia Xu, Xingqian Xu +2
Video generation is experiencing rapid growth, driven by advances in diffusion models and the development of better and larger datasets. However, producing high-quality videos rema…