7 papers
Frozen in a Frame: The Velocity Blind Spot in JEPA World Models
Tinghe Zhang, Chunyu Liu, Yu Leon Liu +6
Joint-embedding predictive architectures (JEPAs) for world modeling train an encoder so a predictor maps a current embedding and action to the next frame's embedding, always from a…
Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL
Qinwei Ma, Jingzhe Shi, Simin Fan +3
ELBO-based reinforcement learning offers a sampler-agnostic approach to fine-tuning flow matching models with reward feedback. Timestep weighting in ELBO-based RL has large impact…
CoMem: Reusing Transformer Depth across Queries with Persistent Intermediate Residuals
Hanzuo Liu, Xuan Qi, Chunyu Liu +6
Repeated queries over shared documents repeatedly execute the same lower transformer layers. We introduce CoMem, which makes split depth j an explicit reusable-context axis: write…
VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination
Chunyu Liu, Zhengyang Fan, Kaisen Yang +1
MDLMs generate text by denoising a preallocated masked response canvas, making response-length modeling central to instruction tuning. Existing MDLMs often inherit the autoregressi…
Improving Sampling for Masked Diffusion Models via Information Gain
Kaisen Yang, Jayden Teoh, Kaicheng Yang +2
Masked Diffusion Models (MDMs) enable flexible decoding orders, yet existing samplers remain largely greedy, selecting locally certain tokens without accounting for their downstrea…
Attention Schema-based Attention Control (ASAC): A Cognitive-Inspired Approach for Attention Management in Transformers
Krati Saxena, Federico Jurado Ruiz, Guido Manzi +2
Attention mechanisms have become integral in AI, significantly enhancing model performance and scalability by drawing inspiration from human cognition. Concurrently, the Attention…
Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm
Kaisen Yang, Tinghe Zhang, Rushi Shah +4
Many LLMs plan before they act, yet planning and execution are often still entangled in one long generation trace, enforced only through prompts, or split across separate component…