9 papers
Chain Of Interaction Benchmark (COIN): When Reasoning meets Embodied Interaction
Xianhao Wang, Xiaojian Ma, Haozhe Hu +7
Generalist embodied agents must perform interactive, causally-dependent reasoning, continually interacting with the environment, acquiring information, and updating plans to solve…
Phys-Diff: A Physics-Inspired Latent Diffusion Model for Tropical Cyclone Forecasting
Lei Liu, Xiaoning Yu, Kang Chen +4
Tropical cyclone (TC) forecasting is critical for disaster warning and emergency response. Deep learning methods address computational challenges but often neglect physical relatio…
MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models
Kailin Jiang, Ning Jiang, Yuntao Du +8
Large Multimodal Models (LMMs) encode rich factual knowledge via cross-modal pre-training, yet their static representations struggle to maintain an accurate understanding of time-s…
KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
Kailin Jiang, Hongbo Jiang, Ning Jiang +7
Large Multimodal Models encode extensive factual knowledge in their pre-trained weights. However, its knowledge remains static and limited, unable to keep pace with real-world deve…
Autoregressive End-to-End Planning with Time-Invariant Spatial Alignment and Multi-Objective Policy Refinement
Jianbo Zhao, Taiyu Ban, Xiangjie Li +5
The inherent sequential modeling capabilities of autoregressive models make them a formidable baseline for end-to-end planning in autonomous driving. Nevertheless, their performanc…
Autoregressive Meta-Actions for Unified Controllable Trajectory Generation
Jianbo Zhao, Taiyu Ban, Xiyang Wang +6
Controllable trajectory generation guided by high-level semantic decisions, termed meta-actions, is crucial for autonomous driving systems. A significant limitation of existing fra…
StyleAR: Customizing Multimodal Autoregressive Model for Style-Aligned Text-to-Image Generation
Yi Wu, Lingting Zhu, Shengju Qian +4
In the current research landscape, multimodal autoregressive (AR) models have shown exceptional capabilities across various domains, including visual understanding and generation.…
When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations
Kailin Jiang, Yuntao Du, Yukai Ding +7
Large Multimodal Models (LMMs) store vast amounts of pretrained knowledge but struggle to remain aligned with real-world updates, making it difficult to avoid capability degradatio…
DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modeling
Jianbo Zhao, Taiyu Ban, Zhihao Liu +7
Accurate and efficient modeling of agent interactions is essential for trajectory generation, the core of autonomous driving systems. Existing methods, scene-centric, agent-centric…