4 papers
Learning When to Abort in Transformer Planning: A Mechanistic Study
Shi Feng, Qimin Zhong, Siwei Wang +4
Planning does not by itself imply the ability to recognise errors, but recognising errors can improve planning capacity. In this paper, we study this phenomenon in Transformer plan…
Looped World Models
Hongyuan Adam Lu, Z. L. Victor Wei, Qun Zhang +28
Current world models face a fundamental tension: faithful long-horizon simulation demands deep computation, but deeper models are expensive to deploy and prone to compounding error…
Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement
Qimin Zhong, Hao Liao, Haiming Qin +4
Whether Large Language Models (LLMs) develop coherent internal world models remains a core debate. While conventional Next-Token Prediction (NTP) focuses on one-step-ahead supervis…
Understanding and Enhancing the Planning Capability of Language Models via Multi-Token Prediction
Qimin Zhong, Hao Liao, Siwei Wang +4
Large Language Models (LLMs) have achieved impressive performance across diverse tasks but continue to struggle with learning transitive relations, a cornerstone for complex planni…