11 papers
SNLP: Layer-Parallel Inference via Structured Newton Corrections
Ligong Han, Kai Xu, Hao Wang +1
Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipeline parallelism. We study w…
Few-Step Diffusion Language Models via Trajectory Self-Distillation
Tunyu Zhang, Xinxi Zhang, Ligong Han +9
Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this poten…
SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
Hao Li, Jingkun An, Zijun Song +8
Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax. Existing methods mitigate this by balancing dual object…
WALL-WM: Carving World Action Modeling at the Event Joints
Shalfun Li, Victor Yao, Charles Yang +28
WALL-WM is a World Action Model that shifts video-action learning from chunk-centric optimization to event-grounded Vision-Language-Action pretraining, using semantically coherent…
Shortcut to Nowhere: Demystifying Deep Spurious Regression
Guanrong Xu, Jessica Li, Hao Wang +1
Real-world regression often exhibits shortcuts: attributes that are spuriously correlated with continuous targets in training, yet unreliable under deployment shifts; regressing ta…
Wall-OSS-0.5 Technical Report
Ryan Yu, Pushi Zhang, Starrick Liu +24
Large-scale Vision-Language-Action (VLA) pretraining is increasingly adopted as the foundation for robot policies, yet the evidence for pretrained VLAs is almost invariably reporte…