Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
arXiv:1502.03143 · doi:10.1109/MRA.2015.2448951
Abstract
In this paper we present the Yale-CMU-Berkeley (YCB) Object and Model set, intended to be used to facilitate benchmarking in robotic manipulation, prosthetic design and rehabilitation research. The objects in the set are designed to cover a wide range of aspects of the manipulation problem; it includes objects of daily life with different shapes, sizes, textures, weight and rigidity, as well as some widely used manipulation tests. The associated database provides high-resolution RGBD scans, physical properties, and geometric models of the objects for easy incorporation into manipulation and planning software platforms. In addition to describing the objects and models in the set along with how they were chosen and derived, we provide a framework and a number of example task protocols, laying out how the set can be used to quantitatively evaluate a range of manipulation approaches including planning, learning, mechanical design, control, and many others. A comprehensive literature survey on existing benchmarks and object datasets is also presented and their scope and limitations are discussed. The set will be freely distributed to research groups worldwide at a series of tutorials at robotics conferences, and will be otherwise available at a reasonable purchase cost. It is our hope that the ready availability of this set along with the ground laid in terms of protocol templates will enable the community of manipulation researchers to more easily compare approaches as well as continually evolve benchmarking tests as the field matures.
Submitted to Robotics and Automation Magazine (RAM) Special Issue on Replicable and Measurable Robotics Research. 35 Pages
Cited by in corpus (69)
- Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
- From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor
- UniGrasp: Learning a Unified Model to Grasp with Multifingered Robotic Hands
- MotionBenchMaker: A Tool to Generate and Benchmark Motion Planning Datasets
- PyRobot: An Open-source Robotics Framework for Research and Benchmarking
- Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
- A study on a Q-Learning algorithm application to a manufacturing assembly problem
- Benchmarking In-Hand Manipulation
- Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger
- Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
- Challenges for Monocular 6D Object Pose Estimation in Robotics
- DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objects
- Multi-modal Transfer Learning for Grasping Transparent and Specular Objects
- Integrated Design and Fabrication of Pneumatic Soft Robot Actuators in a Single Casting Step
- Deep reinforcement learning applied to an assembly sequence planning problem with user preferences
- Household Cloth Object Set: Fostering Benchmarking in Deformable Object Manipulation
- Rigid-Soft Interactive Learning for Robust Grasping
- Fast Object Inertial Parameter Identification for Collaborative Robots
- Benchmarking Robot Manipulation with the Rubik's Cube
- Learning a Decentralized Multi-arm Motion Planner
- Improving the Successful Robotic Grasp Detection Using Convolutional Neural Networks
- DefGraspSim: Simulation-based grasping of 3D deformable objects
- Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images
- Visual Object Recognition in Indoor Environments Using Topologically Persistent Features
- RAMP: A Benchmark for Evaluating Robotic Assembly Manipulation and Planning
- Computational Design of Passive Grippers
- Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation
- A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives
- 6D Object Pose Regression via Supervised Learning on Point Clouds
- Persistent Homology Meets Object Unity: Object Recognition in Clutter
- Deep Tactile Experience: Estimating Tactile Sensor Output from Depth Sensor Data
- DeltaZ: An Accessible Compliant Delta Robot Manipulator for Research and Education
- Markerless visual servoing on unknown objects for humanoid robot platforms
- Communicating human intent to a robotic companion by multi-type gesture sentences
- Ab Initio Particle-based Object Manipulation
- Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
- RF-Annotate: Automatic RF-Supervised Image Annotation of Common Objects in Context
- Inferring 3D Shapes of Unknown Rigid Objects in Clutter through Inverse Physics Reasoning
- PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds
- Weakly-Supervised Object Detection Learning through Human-Robot Interaction
- Event-Driven Visual-Tactile Sensing and Learning for Robots
- Robotic Task Success Evaluation Under Multi-modal Non-Parametric Object Pose Uncertainty
- 3DP3: 3D Scene Perception via Probabilistic Programming
- The ACRV Picking Benchmark (APB): A Robotic Shelf Picking Benchmark to Foster Reproducible Research
- Robust Physics-Based Manipulation by Interleaving Open and Closed-Loop Execution
- CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds
- A Survey on Human Interaction Motion Generation
- Learning a Generative Transition Model for Uncertainty-Aware Robotic Manipulation
- THOR2: Topological Analysis for 3D Shape and Color-Based Human-Inspired Object Recognition in Unseen Environments
- Bayesian optimization for robust robotic grasping using a sensorized compliant hand
- Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework
- Adaptive Grasping of Moving Objects in Dense Clutter via Global-to-Local Detection and Static-to-Dynamic Planning
- BaSeNet: A Learning-based Mobile Manipulator Base Pose Sequence Planning for Pickup Tasks
- Recurrent Connectivity Aids Recognition of Partly Occluded Objects
- Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Net
- A Benchmarking Study of Vision-Based Robotic Grasping Algorithms: A Comparative Analysis
- Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videos
- A dataset of 40K naturalistic 6-degree-of-freedom robotic grasp demonstrations
- Deformation-Aware Data-Driven Grasp Synthesis
- PaMO: Parallel Mesh Optimization for Intersection-Free Low-Poly Modeling on the GPU
- Grasp Synthesis for Novel Objects Using Heuristic-based and Data-driven Active Vision Methods
- A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition
- A Framework for Joint Grasp and Motion Planning in Confined Spaces
- Bayesian Inverse Graphics for Few-Shot Concept Learning
- A Multi-Level Similarity Approach for Single-View Object Grasping: Matching, Planning, and Fine-Tuning
- Attention based visual analysis for fast grasp planning with multi-fingered robotic hand
- Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning
- DeepClaw: A Robotic Hardware Benchmarking Platform for Learning Object Manipulation
- Learning to Predict Contact Force Distributions from Vision Leveraging Object Geometry Priors