Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN

Abstract

To solve the problem of high labour costs in the strawberry picking process, the approach of a strawberry picking robot to identify and find strawberries is suggested in this study. First, 1000 images including mature, immature, single, multiple, and occluded strawberries were collected, and a two-stage detection Mask R-CNN instance segmentation network and a one-stage detection YOLOv3 target detection network were used to train a strawberry identification model which classified strawberries into two categories: mature and immature. The accuracy ratings for YOLOv3 and Mask R-CNN were 93.4% and 94.5%, respectively. Second, the ZED stereo camera, triangulation, and a neural network were used to locate the strawberry in three dimensions. YOLOv3 identification accuracy was 3.1 mm, compared to Mask R-CNN of 3.9 mm. The strawberry detection and positioning method proposed in this study may effectively be used to supply the picking robot with a precise location of the ripe strawberry. Keywords: strawberry detection, 3D point cloud, mean-shift, clustering method DOI: 10.25165/j.ijabe.20221506.7306 Citation: Hu H M, Kaizu Y, Zhang H D, Xu Y W, Imou K, Li M, et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN. Int J Agric & Biol Eng, 2022; 15(6): 175–179.