8 papers · 1 filter
CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
Chensheng Peng, Wenhao Ding, Ran Tian +9
World action models (WAMs) jointly predict actions (intent) and visual future (foresight). Standard training adds noise to recorded actions and video simultaneously, but such train…
RoboSTAR: Next-Scale Autoregressive Sign Language Translation for Humanoid Robots
Yujia Zeng, Chensheng Peng, Yuxin Chen +3
Sign-language interpretation in public communication relies on qualified professional interpreters and can be difficult to scale, motivating robotic signing as a complementary acce…
ECoSim: Data Efficient Fine-Tuning for Controllable Traffic Simulation
Yu-Hsiang Chen, Wei-Jer Chang, Yi-Ting Chen +1
Controllable traffic simulation is critical for testing autonomous driving systems, yet existing approaches often require retraining large generative models with extensive annotate…
Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
Jiuming Liu, Chaojun Ni, Mengmeng Liu +7
With rapid development of large language models and diffusion-based content generation, world modeling has attracted increasing research attention, benefiting various downstream do…
Rethinking Image-to-3D Generation with Sparse Queries: Efficiency, Capacity, and Input-View Bias
Zhiyuan Xu, Jiuming Liu, Yuxin Chen +3
We present SparseGen, a novel framework for efficient image-to-3D generation, which exhibits low input-view bias while being significantly faster. Unlike traditional approaches tha…
URoPE: Universal Relative Position Embedding across Geometric Spaces
Yichen Xie, Depu Meng, Chensheng Peng +4
Relative position embedding has become a standard mechanism for encoding positional information in Transformers. However, existing formulations are typically limited to a fixed geo…
RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space
Yichen Xie, Chensheng Peng, Mazen Abdelfattah +6
World foundation models aim to simulate the evolution of the real world with physically plausible behavior. Unlike prior methods that handle spatial and temporal correlations separ…
StreamDiT: Real-Time Streaming Text-to-Video Generation
Akio Kodaira, Tingbo Hou, Ji Hou +4
Recently, great progress has been achieved in text-to-video (T2V) generation by scaling transformer-based diffusion models to billions of parameters, which can generate high-qualit…