Stereo vision based broccoli recognition and attitude estimation method for field harvesting

Abstract

At present, automatic broccoli harvest in field still faces some issues. It is difficult to segment broccoli in real time under complex field background, and hard to pick tilt-growing broccoli for the end-effector of robot. In this research, an improved YOLOv8n-seg model, named YOLO-Broccoli-Seg was proposed for broccoli recognition. Through adding a triplet attention module to YOLOv8-Seg model, the feature fusion capability of the algorithm is improved significantly. The mean average precision mAP50 (Mask), mAP95 (Mask), mAP50 (Bounding Box, Bbox) and mAP95 (Bbox) of YOLO-Broccoli-Seg are 0.973, 0.683, 0.973 and 0.748 respectively. Precision P -value was improved the most, with an increment of 8.7 %. In addition, an attitude estimation method based on three-dimensional point cloud is proposed. When the tilt angle of broccoli is between −30°and 30°, the R 2 between the estimated value and the true value is 0.934. It indicated that this method can well represent the growth attitude of broccoli. This research can provide the rich broccoli information and technical basis for the automated broccoli picking. • A broccoli recognition algorithm named YOLO-Broccoli-Seg was proposed via adding triplet attention module. • An attitude estimation method based on three-dimensional point cloud is proposed. • This research can provide the rich broccoli information and technical basis for the automated broccoli picking.