8 papers
RePo: Language Models with Context Re-Positioning
Huayang Li, Tianyu Zhao, Deng Cai +1
In-context learning is fundamental to modern Large Language Models (LLMs); however, prevailing architectures impose a rigid and fixed contextual structure by assigning linear or co…
RL-VLA: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Training
Haoran Sun, Yongjian Guo, Zhong Guan +13
Reinforcement learning (RL) has emerged as a critical paradigm for post-training Vision-Language-Action (VLA) models, enabling embodied agents to adapt and improve through environm…
Train Short, Inference Long: Training-free Horizon Extension for Autoregressive Video Generation
Jia Li, Xiaomeng Fu, Xurui Peng +7
Autoregressive video diffusion models have emerged as a scalable paradigm for long video generation. However, they often suffer from severe extrapolation failure, where rapid error…
TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration
Haowei Zhu, Tingxuan Huang, Xing Wang +7
Diffusion models achieve strong generative performance but remain slow at inference due to the need for repeated full-model denoising passes. We present Token-Adaptive Predictor (T…
Fast-weight Product Key Memory
Tianyu Zhao, Llion Jones
Sequence modeling layers in modern language models typically face a trade-off between storage capacity and computational efficiency. While softmax attention offers unbounded storag…
Task-Specific Sparse Feature Masks for Molecular Toxicity Prediction with Chemical Language Models
Kwun Sy Lee, Jiawei Chen, Fuk Sheng Ford Chung +3
Reliable in silico molecular toxicity prediction is a cornerstone of modern drug discovery, offering a scalable alternative to experimental screening. However, the black-box nature…