FCOS-EAM: An accurate segmentation method for overlapping green fruits

摘要

• A clearly structured segmentation framework that applies an edge attention module to focus on features effectively. • New method cooperates the segmentation process of fruit images by two stages of detection followed by segmentation. • BAM module merges the box features to generate the predicted boundary classification scores and boundary locations. • EAM is applied to extract the edge features of the feature map to generate the upsampled feature maps. Accurate fruit detection and segmentation based on deep learning is the key to successful harvesting robot operations, but the complex background of orchards, light and branch shading, and fruit overlap lead to low detection and segmentation accuracy and high complexity of existing methods. To address these problems, an improved green fruit segmentation method based on FCOS is proposed in this study. Firstly, its segmentation function for green fruits is realized by adding segmentation module. Then, the FCOS head network is improved by adding the Border-attention module (BAM) to detect the boundary of green fruits with higher accuracy. In addition, the features of mask branch and edge segmentation branch are fused in the segmentation module, and the appearance commonality is learned by modeling the pairwise affinity between all pixels of the feature map using non-local affinity-parsing, and finally the segmentation prediction results are output by combining the feature map of fruit shape and appearance commonality. The experimental results show that this model achieves 81.2% segmentation accuracy on apple dataset and 77.9% segmentation accuracy on persimmon dataset with relatively low guarantee complexity, which exceeds most current segmentation models. Meanwhile, this model has high robustness and can be used for the detection and segmentation work of other green fruits and vegetables in orchards, while effectively extending the application of agricultural equipment.