Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting

Abstract

In this research, a fully neural network based visual perception framework for autonomous apple harvesting is proposed. The proposed framework includes a multi-function neural network for fruit recognition and a Pointnet grasp estimation to determine the proper grasp pose to guide the robotic execution. Fruit recognition takes raw input of RGB images from the RGB-D camera to perform fruit detection and instance segmentation, and Pointnet grasp estimation take point cloud of each fruit as input and output the prediction of grasp pose for each of fruits. The proposed framework is validated by using RGB-D images collected from laboratory and orchard environments, a robotic grasping test in a controlled environment is also included in the experiments. Experimental shows that the proposed framework can accurately localise and estimate the grasp pose for robotic grasping.

References in corpus (6)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep learning in agriculture: A survey
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• DSSD : Deconvolutional Single Shot Detector
• Single Pixel Reconstruction for One-stage Instance Segmentation

Cited by in corpus (7)

• Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards
• A Survey of Robotic Harvesting Systems and Enabling Technologies
• Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots
• DeepApple: Deep Learning-based Apple Detection using a Suppression Mask R-CNN
• Dataset and Performance Comparison of Deep Learning Architectures for Plum Detection and Robotic Harvesting
• A Real-time Low-cost Artificial Intelligence System for Autonomous Spraying in Palm Plantations
• Review of Fruit Tree Image Segmentation