4 papers
MixFlow Training: Alleviating Exposure Bias with Slowed Interpolation Mixture
Hui Li, Fu-Yun Wang, Haoyuan Xia +4
This paper studies the training-testing discrepancy (a.k.a. exposure bias) problem for improving the diffusion models. During training, the input of a prediction network at one tra…
Unlocking the Address Book: Dissecting the Sparse Semantic Structure of LLM Key-Value Caches via Sparse Autoencoders
Qingsen Ma, Dianyun Wang, Jiaming Lyu +8
The Key-Value (KV) cache is the primary memory bottleneck in long-context Large Language Models, yet it is typically treated as an opaque numerical tensor. In this work, we propose…
ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
Qineng Wang, Wenlong Huang, Yu Zhou +8
Embodied cognition argues that intelligence arises from sensorimotor interaction rather than passive observation. It raises an intriguing question: do modern vision-language models…
AMLA: MUL by ADD in FlashAttention Rescaling
Qichen Liao, Chengqiu Hu, Fangzheng Miao +8
Multi-head Latent Attention (MLA) significantly reduces KVCache memory usage in Large Language Models while introducing substantial computational overhead and intermediate variable…