39 citations · 142 across the 25 of their papers we have counts for
25 papers
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance
Jiayi Guo, Chuanhao Yan, Xingqian Xu +4
Ensuring precise multimodal alignment between diffusion-generated images and input prompts has been a long-standing challenge. Earlier works finetune diffusion weight using high-qu…
RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism
Elia Peruzzo, Dejia Xu, Xingqian Xu +2
Video generation is experiencing rapid growth, driven by advances in diffusion models and the development of better and larger datasets. However, producing high-quality videos rema…
Safe Vision-Language Models via Unsafe Weights Manipulation
Moreno D'Incà, Elia Peruzzo, Xingqian Xu +3
Vision-language models (VLMs) often inherit the biases and unsafe associations present within their large-scale training dataset. While recent approaches mitigate unsafe behaviors,…
GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models
Moreno D'Incà, Elia Peruzzo, Massimiliano Mancini +3
Recent progress in Text-to-Image (T2I) generative models has enabled high-quality image generation. As performance and accessibility increase, these models are gaining significant…
UVMap-ID: A Controllable and Personalized UV Map Generative Model
Weijie Wang, Jichao Zhang, Chang Liu +5
Recently, diffusion models have made significant strides in synthesizing realistic 2D human images based on provided text prompts. Building upon this, researchers have extended 2D…
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
Moreno D'Incà, Elia Peruzzo, Massimiliano Mancini +6
Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply inves…
VASE: Object-Centric Appearance and Shape Manipulation of Real Videos
Elia Peruzzo, Vidit Goel, Dejia Xu +5
Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models. However, most of these methods holistically edit the…
Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
Jiayi Guo, Xingqian Xu, Yifan Pu +6
Recently, diffusion models have made remarkable progress in text-to-image (T2I) generation, synthesizing images with high fidelity and diverse contents. Despite this advancement, l…
Interactive Neural Painting
Elia Peruzzo, Willi Menapace, Vidit Goel +9
In the last few years, Neural Painting (NP) techniques became capable of producing extremely realistic artworks. This paper advances the state of the art in this emerging research…
Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap
Dejia Xu, Xingqian Xu, Wenyan Cong +2
Have you ever imagined how it would look if we placed new objects into paintings? For example, what would it look like if we placed a basketball into Claude Monet's ``Water Lilies,…
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
Xingqian Xu, Jiayi Guo, Zhangyang Wang +3
Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches.…
Zero-shot Generative Model Adaptation via Image-specific Prompt Learning
Jiayi Guo, Chaofei Wang, You Wu +6
Recently, CLIP-guided image synthesis has shown appealing performance on adapting a pre-trained source-domain generator to an unseen target domain. It does not require any target-d…
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models
Eric Zhang, Kai Wang, Xingqian Xu +2
The unlearning problem of deep learning models, once primarily an academic concern, has become a prevalent issue in the industry. The significant advances in text-to-image generati…
PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor
Vidit Goel, Elia Peruzzo, Yifan Jiang +6
Generative image editing has recently witnessed extremely fast-paced growth. Some works use high-level conditioning such as text, while others use low-level conditioning. Neverthel…
Image Completion with Heterogeneously Filtered Spectral Hints
Xingqian Xu, Shant Navasardyan, Vahram Tadevosyan +3
Image completion with large-scale free-form missing regions is one of the most challenging tasks for the computer vision community. While researchers pursue better solutions, drawb…
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xingqian Xu, Zhangyang Wang, Eric Zhang +2
Recent advances in diffusion models have set an impressive milestone in many generation tasks, and trending works such as DALL-E2, Imagen, and Stable Diffusion have attracted great…
Efficient Image Generation with Variadic Attention Heads
Steven Walton, Ali Hassani, Xingqian Xu +2
While the integration of transformers in vision models have yielded significant improvements on vision tasks they still require significant amounts of computation for both training…
Towards Layer-wise Image Vectorization
Xu Ma, Yuqian Zhou, Xingqian Xu +5
Image rasterization is a mature technique in computer graphics, while image vectorization, the reverse path of rasterization, remains a major challenge. Recent advanced deep learni…
VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
Zeyuan Chen, Yinbo Chen, Jingwen Liu +5
Videos typically record the streaming and continuous visual data as discrete consecutive frames. Since the storage cost is expensive for videos of high fidelity, most of them are s…
UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
Xingqian Xu, Zhangyang Wang, Humphrey Shi
The recent success of NeRF and other related implicit neural representation methods has opened a new path for continuous image representation, where pixel values no longer need to…
Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
Xingqian Xu, Zhifei Zhang, Zhaowen Wang +3
Text segmentation is a prerequisite in many real-world text-related tasks, e.g., text style transfer, and scene text removal. However, facing the lack of high-quality datasets and…
The 1st Agriculture-Vision Challenge: Methods and Results
Mang Tik Chiu, Xingqian Xu, Kai Wang +39
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially f…
Deep Affinity Net: Instance Segmentation via Affinity
Xingqian Xu, Mang Tik Chiu, Thomas S. Huang +1
Most of the modern instance segmentation approaches fall into two categories: region-based approaches in which object bounding boxes are detected first and later used in cropping a…