9 papers
Three-Body Scattering for Generative Modeling
Peng Sun, Zhenglin Cheng, Deyuan Liu +3
Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional…
Self-Adversarial One Step Generation via Condition Shifting
Deyuan Liu, Peng Sun, Yansen Han +3
The push for efficient text to image synthesis has moved the field toward one step sampling, yet existing methods still face a three way tradeoff among fidelity, inference speed, a…
Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives
Zecheng Wang, Deyuan Liu, Chunshan Li +5
Standard negative log-likelihood (NLL) for Supervised Fine-Tuning (SFT) applies uniform token-level weighting. This rigidity creates a two-fold failure mode: (i) overemphasizing lo…
Beyond Confidence: The Rhythms of Reasoning in Generative Models
Deyuan Liu, Zecheng Wang, Zhanyue Qin +3
Large Language Models (LLMs) exhibit impressive capabilities yet suffer from sensitivity to slight input context variations, hampering reliability. Conventional metrics like accura…
Efficient Generative Model Training via Embedded Representation Warmup
Deyuan Liu, Peng Sun, Xufeng Li +1
Generative models face a fundamental challenge: they must simultaneously learn high-level semantic concepts (what to generate) and low-level synthesis details (how to generate it).…
Checkpoint Merging via Bayesian Optimization in LLM Pretraining
Deyuan Liu, Zecheng Wang, Bingning Wang +6
The rapid proliferation of large language models (LLMs) such as GPT-4 and Gemini underscores the intense demand for resources during their training processes, posing significant ch…