3 citations · 5 across the 25 of their papers we have counts for
12 papers · 1 filter
When Can Conditional Flow Matching Replace Pointwise Negative Log-Likelihood?
Yansen Han, Hongxin Sun, Tao Lin
Flow matching enables likelihood-free training, yet alignment methods increasingly reuse conditional flow matching (CFM) losses as endpoint negative log-likelihoods (NLLs) and thei…
Three-Body Scattering for Generative Modeling
Peng Sun, Zhenglin Cheng, Deyuan Liu +3
Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional…
Duality Models: An Embarrassingly Simple One-step Generation Paradigm
Peng Sun, Xinyi Shang, Tao Lin +1
Consistency-based generative models like Shortcut and MeanFlow achieve impressive results via a target-aware design for solving the Probability Flow ODE (PF-ODE). Typically, such m…
A Collision-Free Hot-Tier Extension for Engram-Style Conditional Memory: A Controlled Study of Training Dynamics
Tao Lin
We investigate whether high-frequency key collisions are a primary bottleneck in Engram-style conditional memory. To isolate the effect of collisions, we introduce Engram-Nine, a c…
Fast and Stable Diffusion Planning through Variational Adaptive Weighting
Zhiying Qiu, Tao Lin
Diffusion models have recently shown promise in offline RL. However, these methods often suffer from high training costs and slow convergence, particularly when using transformer-b…
Unified Continuous Generative Models
Peng Sun, Yi Jiang, Tao Lin
Recent advances in continuous generative models, including multi-step approaches like diffusion and flow-matching (typically requiring 8-1000 sampling steps) and few-step methods s…
Equally Critical: Samples, Targets, and Their Mappings in Datasets
Runkang Yang, Peng Sun, Xinyi Shang +2
Data inherently possesses dual attributes: samples and targets. For targets, knowledge distillation has been widely employed to accelerate model convergence, primarily relying on t…
Collaborative Unlabeled Data Optimization
Xinyi Shang, Peng Sun, Fengyuan Liu +1
This paper pioneers a novel data-centric paradigm to maximize the utility of unlabeled data, tackling a critical question: How can we enhance the efficiency and sustainability of d…
Efficient Generative Model Training via Embedded Representation Warmup
Deyuan Liu, Peng Sun, Xufeng Li +1
Generative models face a fundamental challenge: they must simultaneously learn high-level semantic concepts (what to generate) and low-level synthesis details (how to generate it).…
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning
Haobo Song, Hao Zhao, Soumajit Majumder +1
Fine-tuning large pre-trained foundation models, such as the 175B GPT-3, has attracted more attention for downstream tasks recently. While parameter-efficient fine-tuning methods h…
Efficiency for Free: Ideal Data Are Transportable Representations
Peng Sun, Yi Jiang, Tao Lin
Data, the seminal opportunity and challenge in modern machine learning, currently constrains the scalability of representation learning and impedes the pace of model evolution. In…
Switch EMA: A Free Lunch for Better Flatness and Sharpness
Siyuan Li, Zicheng Liu, Juanxi Tian +9
Exponential Moving Average (EMA) is a widely used weight averaging (WA) regularization to learn flat optima for better generalizations without extra cost in deep neural network (DN…