Monocular Camera Based Fruit Counting and Mapping with Semantic Data Association

Abstract

We present a cheap, lightweight, and fast fruit counting pipeline that uses a single monocular camera. Our pipeline that relies only on a monocular camera, achieves counting performance comparable to state-of-the-art fruit counting system that utilizes an expensive sensor suite including LiDAR and GPS/INS on a mango dataset. Our monocular camera pipeline begins with a fruit detection component that uses a deep neural network. It then uses semantic structure from motion (SFM) to convert these detections into fruit counts by estimating landmark locations of the fruit in 3D, and using these landmarks to identify double counting scenarios. There are many benefits of developing a low cost and lightweight fruit counting system, including applicability to agriculture in developing countries, where monetary constraints or unstructured environments necessitate cheaper hardware solutions.

arXiv commentAccepted in IEEE Robotics and Automation Letters (RA-L), 8 pages

References in corpus (1)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Cited by in corpus (5)

• Grape detection, segmentation and tracking using deep neural networks and three-dimensional association
• High precision control and deep learning-based corn stand counting algorithms for agricultural robot
• Geomorphological Analysis Using Unpiloted Aircraft Systems, Structure from Motion, and Deep Learning
• Central object segmentation by deep learning for fruits and other roundish objects
• 3D Reconstruction-Based Seed Counting of Sorghum Panicles for Agricultural Inspection