7 papers · 1 filter
Words to World: Benchmarking LLMs in Interactive Physical Environments
Xinnan Dai, Kaiqi Yang, Yaxin Li +9
Words are increasingly becoming an interface for action: large language models (LLMs) can now invoke tools, operate devices, and make decisions that affect the physical world. Howe…
Universal Test-Time Training
Zefan Cai, Qinzhe Hu, Ziqiao Ma +2
Recent Test-Time Training (TTT) architectures compress context into fast weights that are updated online and queried as memory. Existing TTT designs keep this memory private to eac…
Multi-Head Attention Residuals
Cheng Luo, Zefan Cai, Junjie Hu
Transformers propagate information across depth through a single additive residual stream: every sublayer reads only the most recent state. Attention residuals relax this by lettin…
Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers
Chongjian Ge, Hanwen Jiang, Tianyu Wang +9
Visual generation increasingly requires high-resolution images, long videos, and multimodal context, making the quadratic cost of full attention prohibitive. We introduce Chimera,…
UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation
Lin Zhang, Sicheng Mo, Zefan Cai +6
Autoregressive video diffusion models have emerged as a promising approach for long video generation, achieving strong performance in streaming settings. However, existing methods…
Test-Time Training with Next-Token Prediction
Xuan Ouyang, Zefan Cai, Junjie Hu
Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test time. We study whether this signal…
Delta Attention Residuals
Cheng Luo, Zefan Cai, Junjie Hu
Attention Residuals replace standard additive residual connections with learned softmax attention over previous layer outputs, enabling selective cross-layer routing. However, stan…