5 citations · 6 across the 10 of their papers we have counts for
6 papers · 1 filter
Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Models
Kodai Kawamura, Kenji Kawaguchi, Anji Liu
Diffusion language models (DLMs) enable parallel generation by predicting and committing multiple tokens at each denoising step, yet they can generate individually plausible but mu…
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Shuchen Xue, Tianyu Xie, Tianyang Hu +5
Efficiently scaling Large Language Models (LLMs) necessitates exploring alternatives to dominant autoregressive (AR) methods, with Masked Diffusion Models (MDMs) emerging as candid…
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
Yezhen Wang, Zhouhao Yang, Brian K Chen +4
Building upon the success of low-rank adapter (LoRA), low-rank gradient projection (LoRP) has emerged as a promising solution for memory-efficient fine-tuning. However, existing Lo…
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen, Tianyang Hu, Hui Jin +2
In-Context Learning (ICL) has been a powerful emergent property of large language models that has attracted increasing attention in recent years. In contrast to regular gradient-ba…
Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion
Xuantong Liu, Tianyang Hu, Wenjia Wang +2
As a dominant force in text-to-image generation tasks, Diffusion Probabilistic Models (DPMs) face a critical challenge in controllability, struggling to adhere strictly to complex,…
The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling
Jiajun Ma, Shuchen Xue, Tianyang Hu +5
With the incorporation of the UNet architecture, diffusion probabilistic models have become a dominant force in image generation tasks. One key design in UNet is the skip connectio…