1 citations · 1 across the 19 of their papers we have counts for
22 papers
Extreme Compression of Per-Layer Embeddings via Cross-Layer Latent Vector Quantization
Jiacheng Liu, Yaxin Luo, Xiaohan Zhao +5
Scaling embedding capacity alongside Transformer computation is an emerging trend in language models. This trend makes high compression rates increasingly useful. Hence, we study e…
UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation
Deyuan Liu, Yihao Hu, Jingxuan Zhang +13
Dense visual text requires image generators to reproduce long strings across multiple regions with correct placement and legibility. As short-string rendering improves, evaluation…
FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models
Yansen Han, Shengyi Liao, Peng Sun +4
Preference alignment for flow and diffusion models now spans online reinforcement learning and offline preference optimization, but the relation between these methods remains uncle…
LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes
Chuyan Chen, Haoxing Chen, Kun Chen +27
We introduce LLaDA-Image, a unified framework that pairs a 6B Diffusion Transformer (DiT) trained from scratch with a frozen vision-language understanding module built on the LLaDA…
Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training
Peng Sun, Yi Yang, Antong Zhang +7
As supervised fine-tuning data continues to scale, selecting high-value subsets from large candidate pools is crucial for reducing training cost and improving model performance. Ex…
Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training
Peng Sun, Yi Yang, Antong Zhang +7
Data selection in supervised fine-tuning aims to select a small set of effective samples from large-scale candidate data, reducing training cost while preserving model performance.…
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
Chuyan Chen, Peng Sun, Kun Yuan
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the rec…
Three-Body Scattering for Generative Modeling
Peng Sun, Zhenglin Cheng, Deyuan Liu +3
Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional…
Condensing Large-Scale Datasets Directly with Minimal Information Loss
Xinyi Shang, Peng Sun, Bei Shi +2
Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scala…
Self-Adversarial One Step Generation via Condition Shifting
Deyuan Liu, Peng Sun, Yansen Han +3
The push for efficient text to image synthesis has moved the field toward one step sampling, yet existing methods still face a three way tradeoff among fidelity, inference speed, a…
Fast and Scalable Analytical Diffusion
Xinyi Shang, Peng Sun, Jingyu Lin +1
Analytical diffusion models offer a mathematically transparent path to generative modeling by formulating the denoising score as an empirical-Bayes posterior mean. However, this in…
Duality Models: An Embarrassingly Simple One-step Generation Paradigm
Peng Sun, Xinyi Shang, Tao Lin +1
Consistency-based generative models like Shortcut and MeanFlow achieve impressive results via a target-aware design for solving the Probability Flow ODE (PF-ODE). Typically, such m…
TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flows
Zhenglin Cheng, Peng Sun, Jianguo Li +1
Recent advances in large multi-modal generative models have demonstrated impressive capabilities in multi-modal generation, including image and video generation. These models are t…
Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
Ziyu Chen, Xinbei Jiang, Peng Sun +1
Masked Diffusion Models (MDMs) offer flexible, non-autoregressive generation, but this freedom introduces a challenge: final output quality is highly sensitive to the decoding orde…
Unified Continuous Generative Models
Peng Sun, Yi Jiang, Tao Lin
Recent advances in continuous generative models, including multi-step approaches like diffusion and flow-matching (typically requiring 8-1000 sampling steps) and few-step methods s…
Equally Critical: Samples, Targets, and Their Mappings in Datasets
Runkang Yang, Peng Sun, Xinyi Shang +2
Data inherently possesses dual attributes: samples and targets. For targets, knowledge distillation has been widely employed to accelerate model convergence, primarily relying on t…
Collaborative Unlabeled Data Optimization
Xinyi Shang, Peng Sun, Fengyuan Liu +1
This paper pioneers a novel data-centric paradigm to maximize the utility of unlabeled data, tackling a critical question: How can we enhance the efficiency and sustainability of d…
Efficient Generative Model Training via Embedded Representation Warmup
Deyuan Liu, Peng Sun, Xufeng Li +1
Generative models face a fundamental challenge: they must simultaneously learn high-level semantic concepts (what to generate) and low-level synthesis details (how to generate it).…
GMem: A Modular Approach for Ultra-Efficient Generative Models
Yi Tang, Peng Sun, Zhenglin Cheng +1
Recent studies indicate that the denoising process in deep generative diffusion models implicitly learns and memorizes semantic information from the data distribution. These findin…
GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost
Xinyi Shang, Peng Sun, Tao Lin
Recent advancements in dataset distillation have demonstrated the significant benefits of employing soft labels generated by pre-trained teacher models. In this paper, we introduce…
Efficiency for Free: Ideal Data Are Transportable Representations
Peng Sun, Yi Jiang, Tao Lin
Data, the seminal opportunity and challenge in modern machine learning, currently constrains the scalability of representation learning and impedes the pace of model evolution. In…
On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
Peng Sun, Bei Shi, Daiwei Yu +1
Contemporary machine learning requires training large neural networks on massive datasets and thus faces the challenges of high computational demands. Dataset distillation, as a re…