Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot

Abstract

Harvesting grapes requires a large amount of manual labor. To reduce the labor force for the harvesting job, in this study, we developed a robot harvester for the vine grapes. In this paper, we proposed an algorithm that using multi-cameras, as well as artificial intelligence (AI) object detection methods, to detect the thin stem and decide the cut point. The camera system was constructed by two cameras that include multi-lenses. One camera is mounted at the base of the robot and named the "base camera"; the other camera is mounted at the robot hand and named the "hand camera" to recognize grapes and estimate the stem position. At the first step, the grapes are detected by using a You Only Look Once (YOLO) method, while the stems of the grapes are detected at the second step using a pixel-level semantic segmentation method. Field experiments were conducted at an outdoor grapes field. The experiment results show that the proposed algorithm and the camera system can successfully detect out the cut point, and the correct detection rate is around 98% and 93% in the indoor and outdoor conditions, respectively. The detection system was integrated to a grape-harvesting robot in the experiment, and the experiment results show the system can successfully harvest the grapes in the outdoor conditions.