A Precise Detection Method for Tomato Fruit Ripeness and Picking Points in Complex Environments

Abstract

Accurate identification of tomato ripeness and precise detection of picking points is the key to realizing automated picking. Aiming at the problems faced in practical applications, such as low accuracy of tomato ripeness and picking points detection in complex greenhouse environments, which leads to wrong picking, missed picking, and fruit damage by robots, this study proposes the YOLO-TMPPD (Tomato Maturity and Picking Point Detection) model. YOLO-TMPPD is structurally improved and algorithmically optimized based on the YOLOv8 baseline architecture. Firstly, the Depthwise Convolution (DWConv) module is utilized to substitute the C2f module within the backbone network. This substitution not only cuts down the model’s computational load but also simultaneously enhances the detection precision. Secondly, the Content-Aware ReAssembly of FEatures (CARAFE) operator is utilized to enhance the up-sampling operation, enabling precise content-aware processing of tomatoes and picking keypoints to improve accuracy and recall. Finally, the Convolutional Attention Mechanism (CBAM) module is incorporated to enhance the model’s ability to detect tomato-picking key regions in a large field of view in both channel and spatial dimensions. Ablation experiments were conducted to validate the effectiveness of each proposed module (DWConv, CARAFE, CBAM), and the architecture was compared with YOLOv3, v5, v6, v8, v9, and v10. The experimental results reveal that, when juxtaposed with the original network model, the YOLO-TMPPD model brings about remarkable improvements. Specifically, it improves the object detection F1 score by 4.48% and enhances the keypoint detection accuracy by 4.43%. Furthermore, the model’s size is reduced by 8.6%. This study holds substantial theoretical and practical value. In the complex environment of a greenhouse, it contributes significantly to computer-vision-enabled detection of tomato ripening. It can also help robots accurately locate picking points and estimate posture, which is crucial for efficient and precise tomato-picking operations without damage.