A Visual System of Citrus Picking Robot Using Convolutional Neural Networks

Abstract

To realize automatic fruit harvesting, there have been a lot of approaches of engineering since 1960s. However, for the complex natural environment, the study of robotic harvesting systems is still on the developing. In this paper, we propose to use several deep learning methods, which are the-state-of-the-art techniques of pattern recognition, to raise the accuracy of the citrus discrimination by visual sensors. The proposed methods include YOLOv3, ResNet50, and ResNet152, which are the advanced deep convolutional neural networks (CNNs). For the powerful ability of pattern recognition of these CNNS, the proposed visual system is able to distinguish not only citrus fruits but also leaves, branches, and fruits occluded by branch or leaves, and these functions are important for picking work of harvesting robot in the real environment. The recognition abilities of the three CNNs were confirmed by the experiment results, and ResNet152 showed the highest recognition rate. The recognition accuracy of the normal citrus in the natural environment was 95.35%, overlapped citrus fruits reached 97.86%, and 85.12% in the cases of leaves and branches of citrus trees.