29 citations · 70 across the 41 of their papers we have counts for
42 papers
Data Pyramid for Embodied Manipulation: A Survey
Yifan Ye, Yankai Fu, Yaoxu Lv +26
Multimodal foundation models learned to see and to speak by consuming the whole internet. Embodied agents admit no such shortcut, since they require data that couple observations w…
Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
Yunfan Lou, Yifan Ye, Yankai Fu +7
World action models inherit the predictive capability of world models, enabling action generation to be guided by anticipated future observations. However, they rely primarily on v…
Equivalent Flows, Unequal Learning: Clean-Latent Prediction in Transformers
Funing Fu, Tenghui Wang, Guanyu Zhou +2
Flow samplers consume velocity, but the neural network can predict the clean endpoint and convert it to velocity through a fixed affine readout. We study this choice with JLT, a la…
RynnBrain: Open Embodied Foundation Models
Ronghao Dang, Jiayan Guo, Bohan Hou +23
Despite rapid progress in multimodal foundation models, embodied intelligence community still lacks a unified, physically grounded foundation model that integrates perception, reas…
Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
Tianshuai Hu, Xiaolu Liu, Song Wang +17
Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, where hand-crafted interfaces and rule-based components often break down in complex or long-ta…
Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios
Zeyi Liu, Shuang Liu, Jihai Min +7
With the rapid development of industrial intelligence and unmanned inspection, reliable perception and safety assessment for AI systems in complex and dynamic industrial sites has…
Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models
Yifan Ye, Jiaqi Ma, Jun Cen +1
Vision-Language-Action (VLA) models pretrained on large-scale multimodal datasets have emerged as powerful foundations for robotic perception and control. However, their massive sc…
VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
Songqiao Hu, Zeyi Liu, Shuang Liu +5
Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in generalizing across diverse robotic manipulation tasks. However, deploying these models in unstruct…
HiMaCon: Discovering Hierarchical Manipulation Concepts from Unlabeled Multi-Modal Data
Ruizhe Liu, Pei Zhou, Qian Luo +4
Effective generalization in robotic manipulation requires representations that capture invariant patterns of interaction across environments and tasks. We present a self-supervised…
RynnVLA-002: A Unified Vision-Language-Action and World Model
Jun Cen, Siteng Huang, Yuqian Yuan +11
We introduce RynnVLA-002, a unified Vision-Language-Action (VLA) and world model. The world model leverages action and visual inputs to predict future image states, learning the un…
Self-evolved Imitation Learning in Simulated World
Yifan Ye, Jun Cen, Jing Chen +1
Imitation learning has been a trend recently, yet training a generalist agent across multiple tasks still requires large-scale expert demonstrations, which are costly and labor-int…
RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation
Yuming Jiang, Siteng Huang, Shengke Xue +10
This paper presents RynnVLA-001, a vision-language-action(VLA) model built upon large-scale video generative pretraining from human demonstrations. We propose a novel two-stage pre…
Towards Affordance-Aware Robotic Dexterous Grasping with Human-like Priors
Haoyu Zhao, Linghao Zhuang, Xingyue Zhao +10
A dexterous hand capable of generalizable grasping objects is fundamental for the development of general-purpose embodied AI. However, previous methods focus narrowly on low-level…
Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective
Hangjie Yuan, Weihua Chen, Jun Cen +11
Autoregressive large language models (LLMs) have unified a vast range of language tasks, inspiring preliminary efforts in autoregressive (AR) video generation. Existing AR video ge…
WorldVLA: Towards Autoregressive Action World Model
Jun Cen, Chaohui Yu, Hangjie Yuan +9
We present WorldVLA, an autoregressive action world model that unifies action and image understanding and generation. Our WorldVLA intergrates Vision-Language-Action (VLA) model an…
Generative Artificial Intelligence in Robotic Manipulation: A Survey
Kun Zhang, Peng Yun, Jun Cen +11
This survey provides a comprehensive review on recent advancements of generative learning models in robotic manipulation, addressing key challenges in the field. Robotic manipulati…
MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems
Shuhang Chen, Hangjie Yuan, Yunqiu Xu +5
Despite strong results on many tasks, multimodal large language models (MLLMs) still underperform on visual mathematical problem solving, especially in reliably perceiving and inte…
LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving
Lingdong Kong, Xiang Xu, Youquan Liu +6
Recent advancements in vision foundation models (VFMs) have revolutionized visual perception in 2D, yet their potential for 3D scene understanding, particularly in autonomous drivi…
Continual Learning for Segment Anything Model Adaptation
Jinglong Yang, Yichen Wu, Jun Cen +3
Although the current different types of SAM adaptation methods have achieved promising performance for various downstream tasks, such as prompt-based ones and adapter-based ones, m…
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
Chenyang Lei, Liyi Chen, Jun Cen +5
Foundation models like ChatGPT and Sora that are trained on a huge scale of data have made a revolutionary social impact. However, it is extremely challenging for sensors in many d…
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality
Chenyang Lei, Liyi Chen, Jun Cen +6
Foundation models like ChatGPT and Sora that are trained on a huge scale of data have made a revolutionary social impact. However, it is extremely challenging for sensors in many d…
iControl3D: An Interactive System for Controllable 3D Scene Generation
Xingyi Li, Yizheng Wu, Jun Cen +6
3D content creation has long been a complex and time-consuming process, often requiring specialized skills and resources. While recent advancements have allowed for text-guided 3D…
4D Panoptic Scene Graph Generation
Jingkang Yang, Jun Cen, Wenxuan Peng +6
We are living in a three-dimensional space while moving forward through a fourth dimension: time. To allow artificial intelligence to develop a comprehensive understanding of such…
Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations
Kewei Wang, Yizheng Wu, Jun Cen +5
The perception of motion behavior in a dynamic environment holds significant importance for autonomous driving systems, wherein class-agnostic motion prediction methods directly pr…