Visual Tracking Method of Tomato Plant Main-Stems for Robotic Harvesting

Abstract

Robotic harvesting was greatly needed as the raising labor cost on tomato production in China. However, it is difficult to realizing the efficient picking operation as the fruits randomly grow along the tall plant main-stem. In order to improve the fruit detection efficiency and expand robot working space, the visual tracking method of tomato main-stem was proposed, which was considered as a support for fruits' active detection. The Mask RCNN model was adopted to identify the plant's main-stem through transfer learning, and its centerline was located according to the moment feature of the identified mask area. The control method for the pan-tilt camera based on visual servo was proposed, so that the camera's posture could be automatically adjusted to scan and search along the main-stem. The field test results showed that, the Mask RCNN model's error rate, accuracy rate and recall rate respectively was 0.21, 093 and 0.80 for main-stem segmentation, and the locating deviation of tracking point in various view-field averagely was 21 pixels, which was little enough to ensure the image center keep in the main-stem area during tracking.