Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots

Abstract

Vision perception and modelling are the essential tasks of robotic harvesting in the unstructured orchard. This paper develops a framework of visual perception and modelling for robotic harvesting of fruits in the orchard environments. The developed framework includes visual perception, scenarios mapping, and fruit modelling. The Visual perception module utilises a deep-learning model to perform multi-purpose visual perception task within the working scenarios; The scenarios mapping module applies OctoMap to represent the multiple classes of objects or elements within the environment; The fruit modelling module estimates the geometry property of objects and estimates the proper access pose of each fruit. The developed framework is implemented and evaluated in the apple orchards. The experiment results show that visual perception and modelling algorithm can accurately detect and localise the fruits, and modelling working scenarios in real orchard environments. The $F_1$ score and mean intersection of union of visual perception module on fruit detection and segmentation are 0.833 and 0.852, respectively. The accuracy of the fruit modelling in terms of centre localisation and pose estimation are 0.955 and 0.923, respectively. Overall, an accurate visual perception and modelling algorithm are presented in this paper.

21 pages,10 figures

References in corpus (7)

• Deep learning in agriculture: A survey
• Fully Convolutional Networks for Semantic Segmentation
• Autonomous Sweet Pepper Harvesting for Protected Cropping Systems
• Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards
• Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting
• Fast Autonomous Flight in Warehouses for Inventory Applications
• Single Pixel Reconstruction for One-stage Instance Segmentation

Cited by in corpus (4)

• Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards
• A Survey of Robotic Harvesting Systems and Enabling Technologies
• Accurate Fruit Localisation for Robotic Harvesting using High Resolution LiDAR-Camera Fusion
• Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchards