12 papers
DepState-JEPA: Learning Depth-Grounded State Representation for Visual Perception and Manipulation
Xiujian Liang, Liuyuan Na, Zehao Du +6
Predictive representations for robot manipulation should connect scene geometry with the state changes induced by actions. Our intuition is that understanding what an action will c…
Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models
Mingyang Sun, Jiude Wei, Xiujian Liang +4
Current Vision-Language-Action (VLA) models rely mainly on 2D inputs, neglecting the rich object structural information and commonsense knowledge inherent in the 3D physical world.…
Analytic Concept-Centric Memory for Agentic Embodied Manipulation
Mingyang Sun, Xiujian Liang, Jiude Wei +4
Long-horizon embodied manipulation requires agents to remember persistent objects, track changing scene states, and reuse prior interaction knowledge. However, existing agent memor…
OptArgus: A Multi-Agent System to Detect Hallucinations in LLM-based Optimization Modeling
Zhong Li, Zihan Guo, Xiaohan Lu +5
Large language models (LLMs) are increasingly used to translate natural-language optimization problems into mathematical formulations and solver code, but matching the reference ob…
Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning
Mingyang Sun, Pengxiang Ding, Weinan Zhang +1
While behavior cloning with flow/diffusion policies excels at learning complex skills from demonstrations, it remains vulnerable to distributional shift, and standard RL methods st…
Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation
Mingyang Sun, Jiude Wei, Qichen He +3
Enabling robots to perform precise and generalized manipulation in unstructured environments remains a fundamental challenge in embodied AI. While Vision-Language Models (VLMs) hav…
VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
Hengtao Li, Pengxiang Ding, Runze Suo +8
Vision-Language-Action (VLA) models enable embodied decision-making but rely heavily on imitation learning, leading to compounding errors and poor robustness under distribution shi…
SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
Yang Liu, Ming Ma, Xiaomin Yu +5
Despite impressive advancements in Visual-Language Models (VLMs) for multi-modal tasks, their reliance on RGB inputs limits precise spatial understanding. Existing methods for inte…
Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport
Mingyang Sun, Pengxiang Ding, Weinan Zhang +1
Diffusion policies have shown promise in learning complex behaviors from demonstrations, particularly for tasks requiring precise control and long-term planning. However, they face…
Score and Distribution Matching Policy: Advanced Accelerated Visuomotor Policies via Matched Distillation
Bofang Jia, Pengxiang Ding, Can Cui +5
Visual-motor policy learning has advanced with architectures like diffusion-based policies, known for modeling complex robotic trajectories. However, their prolonged inference time…
QUART-Online: Latency-Free Large Multimodal Language Model for Quadruped Robot Learning
Xinyang Tong, Pengxiang Ding, Yiguo Fan +9
This paper addresses the inherent inference latency challenges associated with deploying multimodal large language models (MLLM) in quadruped vision-language-action (QUAR-VLA) task…
CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Prediction
Zhefei Gong, Pengxiang Ding, Shangke Lyu +5
In robotic visuomotor policy learning, diffusion-based models have achieved significant success in improving the accuracy of action trajectory generation compared to traditional au…