30 citations · 148 across the 82 of their papers we have counts for
21 papers · 1 filter
UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models
Jiaqi Wang, Haoge Deng, Ting Pan +5
Uniform Discrete Diffusion Model (UDM) has recently emerged as a promising paradigm for discrete generative modeling; however, its integration with reinforcement learning remains l…
Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
Shezheng Song, Shasha Li, Shan Zhao +6
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language understanding, yet how they internally integrate visual and textual information remain…
Emu3.5: Native Multimodal Models are World Learners
Yufeng Cui, Honghao Chen, Haoge Deng +20
We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language. Emu3.5 is pre-trained end-to-end with a unified next-tok…
Shared Neural Space: Unified Precomputed Feature Encoding for Multi-Task and Cross Domain Vision
Jing Li, Oskar Bartosz, Chengyu Wang +3
The majority of AI models in imaging and vision are customized to perform on specific high-precision task. However, this strategy is inefficient for applications with a series of m…
Encapsulated Composition of Text-to-Image and Text-to-Video Models for High-Quality Video Synthesis
Tongtong Su, Chengyu Wang, Bingyan Liu +2
In recent years, large text-to-video (T2V) synthesis models have garnered considerable attention for their abilities to generate videos from textual descriptions. However, achievin…
Video Event Reasoning and Prediction by Fusing World Knowledge from LLMs with Vision Foundation Models
L'ea Dubois, Klaus Schmidt, Chengyu Wang +3
Current video understanding models excel at recognizing "what" is happening but fall short in high-level cognitive tasks like causal reasoning and future prediction, a limitation r…
MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images
Chengyu Wang, Isabella Bennett, Henry Scott +4
We present MatDecompSDF, a novel framework for recovering high-fidelity 3D shapes and decomposing their physically-based material properties from multi-view images. The core challe…
Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing
Tongtong Su, Chengyu Wang, Jun Huang +1
Appearance editing according to user needs is a pivotal task in video editing. Existing text-guided methods often lead to ambiguities regarding user intentions and restrict fine-gr…
Understanding Attention Mechanism in Video Diffusion Models
Bingyan Liu, Chengyu Wang, Tongtong Su +4
Text-to-video (T2V) synthesis models, such as OpenAI's Sora, have garnered significant attention due to their ability to generate high-quality videos from a text prompt. In diffusi…
MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image
Shezheng Song, Chengxiang He, Shan Zhao +4
Multimodal large language models (MLLMs) have shown remarkable progress in high-level semantic tasks such as visual question answering, image captioning, and emotion recognition. H…
DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding
Wenhui Liao, Jiapeng Wang, Hongliang Li +3
Text-rich document understanding (TDU) requires comprehensive analysis of documents containing substantial textual content and complex layouts. While Multimodal Large Language Mode…
Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing
Bingyan Liu, Chengyu Wang, Tingfeng Cao +2
Deep Text-to-Image Synthesis (TIS) models such as Stable Diffusion have recently gained significant popularity for creative Text-to-image generation. Yet, for domain-specific scena…
Diffutoon: High-Resolution Editable Toon Shading via Diffusion Models
Zhongjie Duan, Chengyu Wang, Cen Chen +2
Toon shading is a type of non-photorealistic rendering task of animation. Its primary purpose is to render objects with a flat and stylized appearance. As diffusion models have asc…
A Contrastive Compositional Benchmark for Text-to-Image Synthesis: A Study with Unified Text-to-Image Fidelity Metrics
Xiangru Zhu, Penglei Sun, Chengyu Wang +4
Text-to-image (T2I) synthesis has recently achieved significant advancements. However, challenges remain in the model's compositionality, which is the ability to create new combina…
FastBlend: a Powerful Model-Free Toolkit Making Video Stylization Easier
Zhongjie Duan, Chengyu Wang, Cen Chen +3
With the emergence of diffusion models and rapid development in image processing, it has become effortless to generate fancy images in tasks such as style transfer and image editin…
DualToken-ViT: Position-aware Efficient Vision Transformer with Dual Token Fusion
Zhenzhen Chu, Jiayu Chen, Cen Chen +4
Self-attention-based vision transformers (ViTs) have emerged as a highly competitive architecture in computer vision. Unlike convolutional neural networks (CNNs), ViTs are capable…
DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis
Zhongjie Duan, Lizhou You, Chengyu Wang +4
In recent years, diffusion models have emerged as the most powerful approach in image synthesis. However, applying these models directly to video synthesis presents challenges, as…
ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval
Jiapeng Wang, Chengyu Wang, Xiaodan Wang +2
Large-scale pre-trained text-image models with dual-encoder architectures (such as CLIP) are typically adopted for various vision-language applications, including text-image retrie…
Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Models
Zhongjie Duan, Chengyu Wang, Cen Chen +2
In recent years, diffusion models have become the most popular and powerful methods in the field of image synthesis, even rivaling human artists in artistic creativity. However, th…
INTERN: A New Learning Paradigm Towards General Vision
Jing Shao, Siyu Chen, Yangguang Li +24
Enormous waves of technological innovations over the past several years, marked by the advances in AI technologies, are profoundly reshaping the industry and the society. However,…
Video Generation from Single Semantic Label Map
Junting Pan, Chengyu Wang, Xu Jia +4
This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides a good balance between flexibility and quality in the generation p…