32 citations · 44 across the 22 of their papers we have counts for
24 papers
Exploring Spatial Intelligence from a Generative Perspective
Muzhi Zhu, Shunyao Jiang, Huanyi Zheng +9
Spatial intelligence is essential for multimodal large language models, yet current benchmarks largely assess it only from an understanding perspective. We ask whether modern gener…
MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation
Liyang Li, Wen Wang, Canyu Zhao +4
Recent advances in Diffusion Transformers (DiTs) have enabled high-quality joint audio-video generation, producing videos with synchronized audio within a single model. However, ex…
CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models
Junming Huang, Chi Wang, Letian Li +5
Large Language Models(LLMs) have revolutionized text generation and multimodal perception,but their capabilities in 3D content generation remain underexplored. Existing methods com…
SurfaceSplat: Connecting Surface Reconstruction and Gaussian Splatting
Zihui Gao, Jia-Wang Bian, Guosheng Lin +2
Surface reconstruction and novel view rendering from sparse-view images are challenging. Signed Distance Function (SDF)-based methods struggle with fine details, while 3D Gaussian…
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories
Muzhi Zhu, Yuzhuo Tian, Hao Chen +5
While MLLMs have demonstrated adequate image understanding capabilities, they still struggle with pixel-level comprehension, limiting their practical applications. Current evaluati…
A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction
Yongfan Chen, Xiuwen Zhu, Tianyu Li
Recent advances in video generation models demonstrate their potential as world simulators, but they often struggle with videos deviating from physical laws, a key concern overlook…
Revisiting Convolution Architecture in the Realm of DNA Foundation Models
Yu Bo, Weian Mao, Yanjun Shao +6
In recent years, a variety of methods based on Transformer and state space model (SSM) architectures have been proposed, advancing foundational DNA language models. However, there…
DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks
Canyu Zhao, Yanlong Sun, Mingyu Liu +6
This paper's primary objective is to develop a robust generalist perception model capable of addressing multiple tasks under constraints of computational resources and limited trai…
REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observation
Xizhe Xue, Guoting Wei, Hao Chen +4
The rapid evolution of Vision Language Models (VLMs) has catalyzed significant advancements in artificial intelligence, expanding research across various disciplines, including Ear…
Framer: Interactive Frame Interpolation
Wen Wang, Qiuyu Wang, Kecheng Zheng +6
We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking t…
TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddings
Dawei Yan, Pengcheng Li, Yang Li +7
Currently, inspired by the success of vision-language models (VLMs), an increasing number of researchers are focusing on improving VLMs and have achieved promising results. However…
FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior
Zhekai Chen, Wen Wang, Zhen Yang +3
We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. Rather than concentrating on specific use cases such as appear…
MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequence
Canyu Zhao, Mingyu Liu, Wen Wang +5
Recent advancements in video generation have primarily leveraged diffusion models for short-duration content. However, these approaches often fall short in modeling complex narrati…
Generative Active Learning for Long-tailed Instance Segmentation
Muzhi Zhu, Chengxiang Fan, Hao Chen +4
Recently, large-scale language-image generative models have gained widespread attention and many works have utilized generated data from these models to further enhance the perform…
FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition
Ganggui Ding, Canyu Zhao, Wen Wang +4
Benefiting from large-scale pre-trained text-to-image (T2I) generative models, impressive progress has been achieved in customized image generation, which aims to generate user-spe…
DiverGen: Improving Instance Segmentation by Learning Wider Data Distribution with More Diverse Generative Data
Chengxiang Fan, Muzhi Zhu, Hao Chen +4
Instance segmentation is data-hungry, and as model capacity increases, data scale becomes crucial for improving the accuracy. Most instance segmentation datasets today require cost…
Zippo: Zipping Color and Transparency Distributions into a Single Diffusion Model
Kangyang Xie, Binbin Yang, Hao Chen +5
Beyond the superiority of the text-to-image diffusion model in generating high-quality images, recent studies have attempted to uncover its potential for adapting the learned seman…
Diffusion Models are Efficient Data Generators for Human Mesh Recovery
Yongtao Ge, Wenjia Wang, Yongfan Chen +4
Despite remarkable progress having been made on the problem of 3D human pose and shape estimation (HPS), current state-of-the-art methods rely heavily on either confined indoor moc…
GenDeF: Learning Generative Deformation Field for Video Generation
Wen Wang, Kecheng Zheng, Qiuyu Wang +5
We offer a new perspective on approaching the task of video generation. Instead of directly synthesizing a sequence of frames, we propose to render a video by warping one static im…
AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort
Wen Wang, Canyu Zhao, Hao Chen +3
Story visualization aims to generate a series of images that match the story described in texts, and it requires the generated images to satisfy high quality, alignment with the te…
De novo protein design using geometric vector field networks
Weian Mao, Muzhi Zhu, Zheng Sun +4
Innovations like protein diffusion have enabled significant progress in de novo protein design, which is a vital topic in life science. These methods typically depend on protein st…
Object-aware Inversion and Reassembly for Image Editing
Zhen Yang, Ganggui Ding, Wen Wang +3
By comparing the original and target prompts, we can obtain numerous editing pairs, each comprising an object and its corresponding editing target. To allow editability while maint…
RGM: A Robust Generalizable Matching Model
Songyan Zhang, Xinyu Sun, Hao Chen +2
Finding corresponding pixels within a pair of images is a fundamental computer vision task with various applications. Due to the specific requirements of different tasks like optic…
DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models
Weijia Wu, Yuzhong Zhao, Hao Chen +6
Current deep networks are very data-hungry and benefit from training on largescale datasets, which are often time-consuming to collect and annotate. By contrast, synthetic data can…