78 citations · 177 across the 32 of their papers we have counts for
32 papers
DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformers
Binglei Li, Mengping Yang, Zhiyu Tan +4
Recent advances in Diffusion Transformers (DiTs) have enabled remarkable progress in visual synthesis, benefiting from their superior scalability. To facilitate DiTs' capability of…
Monkey King Bang: A Unified Scientific Multimodal Foundation Model
Hesen Chen, Xinyu Su, Xiaomeng Yang +11
Scientific discovery is increasingly shifting from isolated disciplines to multi-domain reasoning, and AI for science faces a similar transition. Existing systems are either specia…
PhyMRI-SR: Toward Physics-Aware MRI Image Super-Resolution
Lihua Wei, Huatong Gao, Jia Gong +4
Magnetic resonance imaging (MRI) super-resolution is vital for improving diagnostic accessibility, yet most methods treat it as a deterministic mapping from a fixed low-resolution…
Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator
Luozheng Qin, Jia Gong, Qian Qiao +6
Unified multimodal models integrating visual understanding and generation face a fundamental challenge: visual generation incurs substantially higher computational costs than under…
Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing
Hao Yang, Zhiyu Tan, Jia Gong +7
We present Omni-Video 2, a scalable and computationally efficient model that connects pretrained multimodal large-language models (MLLMs) with video diffusion models for unified vi…
DiverseDiT: Towards Diverse Representation Learning in Diffusion Transformers
Mengping Yang, Zhiyu Tan, Binglei Li +3
Recent breakthroughs in Diffusion Transformers (DiTs) have revolutionized the field of visual synthesis due to their superior scalability. To facilitate DiTs' capability of capturi…
Diff-Aid: Inference-time Adaptive Interaction Denoising for Rectified Text-to-Image Generation
Binglei Li, Mengping Yang, Zhiyu Tan +2
Recent text-to-image (T2I) diffusion models have achieved remarkable advancement, yet faithfully following complex textual descriptions remains challenging due to insufficient inte…
A unified multimodal understanding and generation model for cross-disciplinary scientific research
Xiaomeng Yang, Zhiyu Tan, Xiaohui Zhong +5
Scientific discovery increasingly relies on integrating heterogeneous, high-dimensional data across disciplines nowadays. While AI models have achieved notable success across vario…
TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers
Binglei Li, Mengping Yang, Zhiyu Tan +2
Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and Qwen Image, have facilitated thr…
Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Vision
Luozheng Qin, Jia Gong, Yuqing Sun +6
Chain-of-Thought (CoT) reasoning has been widely adopted to enhance Large Language Models (LLMs) by decomposing complex tasks into simpler, sequential subtasks. However, extending…
Omni-Video: Democratizing Unified Video Understanding and Generation
Zhiyu Tan, Hao Yang, Luozheng Qin +3
Notable breakthroughs in unified understanding and generation modeling have led to remarkable advancements in image understanding, reasoning, production and editing, yet current fo…
RDPO: Real Data Preference Optimization for Physics Consistency Video Generation
Wenxu Qian, Chaoyue Wang, Hou Peng +3
Video generation techniques have achieved remarkable advancements in visual quality, yet faithfully reproducing real-world physics remains elusive. Preference-based model post-trai…
SIPO: Stabilized and Improved Preference Optimization for Aligning Diffusion Models
Xiaomeng Yang, Mengping Yang, Junyan Wang +3
Preference learning has garnered extensive attention as an effective technique for aligning diffusion models with human preferences in visual generation. However, existing alignmen…
Mirror Descent Linearized Augmented Lagrangian Methods for Nonconvex Constrained Stochastic Zeroth-Order Optimization
Qiankun Shi, Han Yuan, Xiao Wang +1
In this paper, we study nonconvex constrained stochastic zeroth-order optimization problems with exact constraints and stochastic objective evaluations. To solve this class of prob…
Cockatiel: Ensembling Synthetic and Human Preferenced Training for Detailed Video Caption
Luozheng Qin, Zhiyu Tan, Mengping Yang +2
Video Detailed Captioning (VDC) is a crucial task for vision-language bridging, enabling fine-grained descriptions of complex video content. In this paper, we first comprehensively…
SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models
Hesen Chen, Junyan Wang, Zhiyu Tan +1
Modern diffusion models encounter a fundamental trade-off between training efficiency and generation quality. While existing representation alignment methods, such as REPA, acceler…
Raccoon: Multi-stage Diffusion Training with Coarse-to-Fine Curating Videos
Zhiyu Tan, Junyan Wang, Hao Yang +4
Text-to-video generation has demonstrated promising progress with the advent of diffusion models, yet existing approaches are limited by dataset quality and computational resources…
Dual-IPO: Dual-Iterative Preference Optimization for Text-to-Video Generation
Xiaomeng Yang, Mengping Yang, Jia Gong +3
Recent advances in video generation have enabled thrilling experiences in producing realistic videos driven by scalable diffusion transformers. However, they usually fail to produc…
E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models
Zhiyu Tan, WenXu Qian, Hesen Chen +3
Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling, revolutionizing the field through remarkable success across various div…
LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment
Yibin Wang, Zhiyu Tan, Junyan Wang +3
Recent advances in text-to-video (T2V) generative models have shown impressive capabilities. However, these models are still inadequate in aligning synthesized videos with human pr…
ReToMe-VA: Recursive Token Merging for Video Diffusion-based Unrestricted Adversarial Attack
Ziyi Gao, Kai Chen, Zhipeng Wei +5
Recent diffusion-based unrestricted attacks generate imperceptible adversarial examples with high transferability compared to previous unrestricted attacks and restricted attacks.…
VidGen-1M: A Large-Scale Dataset for Text-to-video Generation
Zhiyu Tan, Xiaomeng Yang, Luozheng Qin +1
The quality of video-text pairs fundamentally determines the upper bound of text-to-video models. Currently, the datasets used for training these models suffer from significant sho…
EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models
Zhiyu Tan, Xiaomeng Yang, Luozheng Qin +3
The recent advancements in text-to-image generative models have been remarkable. Yet, the field suffers from a lack of evaluation metrics that accurately reflect the performance of…
An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual Representation
Zhiyu Tan, Mengping Yang, Luozheng Qin +5
One critical prerequisite for faithful text-to-image generation is the accurate understanding of text inputs. Existing methods leverage the text encoder of the CLIP model to repres…