Single RGB-D Camera Teleoperation for General Robotic Manipulation
arXiv:2106.14396
Abstract
We propose a teleoperation system that uses a single RGB-D camera as the human motion capture device. Our system can perform general manipulation tasks such as cloth folding, hammering and 3mm clearance peg in hole. We propose the use of non-Cartesian oblique coordinate frame, dynamic motion scaling and reposition of operator frames to increase the flexibility of our teleoperation system. We hypothesize that lowering the barrier of entry to teleoperation will allow for wider deployment of supervised autonomy system, which will in turn generates realistic datasets that unlock the potential of machine learning for robotic manipulation. Demo of our systems are available online https://sites.google.com/view/manipulation-teleop-with-rgbd
References in corpus (8)
- Language Models are Few-Shot Learners
- Embodied Hands: Modeling and Capturing Hands and Bodies Together
- MediaPipe Hands: On-device Real-time Hand Tracking
- FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration
- Visual Imitation Made Easy
- How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?
- Human-in-the-Loop Imitation Learning using Remote Teleoperation
- Auto-Tuned Sim-to-Real Transfer