HOB-CNNv2: Deep learning based detection of extremely occluded tree branches and reference to the dominant tree image

Abstract

Against the backdrop of a global labour shortage, the cost of agriculture has risen rapidly. Using robots to replace manual orchard maintenance tasks has attracted more attention. To avoid collisions between the robot and the tree canopy, a reliable vision system that can detect tree branches under natural occlusions is critical for robot navigation. In this paper, a regression deep learning based vision model, HOB-CNNv2, is proposed for the detection of continuous tree branches under natural occlusions in summer. The model is tested under two occlusion conditions, heavily occluded and extremely occluded. The experimental results show that HOB-CNNv2 can accurately detect tree branches in both occlusion conditions and outperforms the state-of-the-art semantic segmentation model, U-Net, in terms of branch position and thickness accuracy. To our best knowledge, this is the first vision model designed to deal with extremely occluded trees. In addition, we investigated five different methods using paired winter tree images to improve the performance of HOB-CNNv2. The results show that the best method is to use the winter and summer images of the same tree as two inputs to the neural network to extract features separately. This suggests that the model potentially acquires distinct knowledge or patterns from the winter and summer images.