8 citations · 13 across the 22 of their papers we have counts for
21 papers · 1 filter
Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation
Wenqiao Li, Qianyou Zhao, Jiawen Hao +5
Dexterous manipulation requires tactile feedback. However, robot tactile demonstrations are difficult to scale,because dexterous-hand teleoperation provides limited tactile feedbac…
Learning In-Hand Object Reaching to General 6D Poses
Junxiao Lin, Tianyue Wu, Jie Yin +3
In-hand manipulation allows multi-fingered dexterous hands to reconfigure grasped objects without releasing and regrasping them. This improves manipulation efficiency by reducing r…
Grounding Generated Video Plans in Simulation Towards Versatile Dexterous Controllers
Tianyue Wu, Boyuan An, Shuqi Zhao +6
Generated hand-object interaction (HOI) videos provide a controllable way to propose manipulation motions. Simulation-based HOI tracking can translate such kinematic references int…
Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
Ruoqu Chen, Feixiang Ruan, Liu Cao +9
Human videos are an abundant source of dexterous manipulation behaviors, but they lack tactile information that is crucial for contact-rich interaction. This raises a fundamental q…
R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models
Yidi Wang, Feixiang Ruan, Ruoqu Chen +4
Evaluating robot manipulation policies is becoming increasingly important as generalist models, particularly vision-language-action (VLA) models, are deployed on physical robots. H…
Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
Guanxiong Chen, Qianjun Xia, Jiawei Peng +24
Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover…
BoxTwin: Learning Elastoplastic Articulated Object Dynamics from Videos
Heng Zhang, Gehan Zheng, Kaifeng Zhang +6
Digital twins enable robots to anticipate and adapt to physical interactions, but existing models struggle with elastoplastic articulated objects (EAOs) that exhibit nonlinear elas…
Learning Physics-Guided Residual Dynamics for Deformable Object Simulation
Shivansh Patel, Kaifeng Zhang, Sanjay Pokkali +2
Simulating deformable objects is essential for a wide range of robotic manipulation applications, yet accurately predicting their dynamics remains challenging. We propose Physics-G…
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
Chengbo Yuan, Zicheng Zhang, Mingjie Zhou +14
Despite the success of vision-based generalist robotic policies, existing tactile-based policies remain tied to fixed embodiments and sensor setups. This is because tactile signals…
UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms
Yufei Jia, Zhanxiang Cao, Mingrui Yu +48
Simulation-based RL for contemporary robot control is increasingly organized around GPU-resident simulation: physics, rollout collection, and learning are placed on a single GPU-ce…
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
Tutian Tang, Xingyu Ji, Wanli Xing +7
While Vision-Language-Action (VLA) models have demonstrated remarkable success in robotic manipulation, their application has largely been confined to low-degree-of-freedom end-eff…
DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization
Qianyou Zhao, Wenqiao Li, Chiyu Wang +1
High-fidelity teleoperation of dexterous robotic hands is essential for bringing robots into unstructured domestic environments. However, existing teleoperation systems often face…
ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
Yiting Chen, Kenneth Kimble, Edward H. Adelson +20
Dexterous manipulation enables robots to purposefully alter the physical world, transforming them from passive observers into active agents in unstructured environments. This capab…
Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map
Lei Su, Zhijie Peng, Renyuan Ren +4
Vision-Based Tactile Sensors (VBTS) are essential for achieving dexterous robotic manipulation, yet the tactile sim-to-real gap remains a fundamental bottleneck. Current tactile si…
Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
Kaifeng Zhang, Shuo Sha, Hanxiao Jiang +7
Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for task…
Spatially anchored Tactile Awareness for Robust Dexterous Manipulation
Jialei Huang, Yang Ye, Yuanqing Gong +3
Dexterous manipulation requires precise geometric reasoning, yet existing visuo-tactile learning methods struggle with sub-millimeter precision tasks that are routine for tradition…
ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation
Liang Heng, Haoran Geng, Kaifeng Zhang +2
Dexterous manipulation is a cornerstone capability for robotic systems aiming to interact with the physical world in a human-like manner. Although vision-based methods have advance…
Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videos
Kaifeng Zhang, Baoyu Li, Kris Hauser +1
Modeling the dynamics of deformable objects is challenging due to their diverse physical properties and the difficulty of estimating states from limited visual information. We addr…
Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling
Mingtong Zhang, Kaifeng Zhang, Yunzhu Li
Videos of robots interacting with objects encode rich information about the objects' dynamics. However, existing video prediction approaches typically do not explicitly account for…
AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation
Kaifeng Zhang, Baoyu Li, Kris Hauser +1
Predictive models are a crucial component of many robotic systems. Yet, constructing accurate predictive models for a variety of deformable objects, especially those with unknown p…
Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance
Kaifeng Zhang, Zhao-Heng Yin, Weirui Ye +1
Defining reward functions for skill learning has been a long-standing challenge in robotics. Recently, vision-language models (VLMs) have shown promise in defining reward signals f…