BBNeuS: Segmentation and accurate 3D reconstruction of banana bunches from complex plantation environments

Abstract

Accurate 3D reconstruction provides essential spatial information for orchard robots and serves as a critical foundation for crop phenotypic analysis. However, most existing studies have focused on industrial scenarios and are typically conducted in interference-free indoor environments. In this study, we propose a novel 3D reconstruction method called BBNeuS, which achieves accurate reconstruction of banana bunches in real-world orchard conditions. To accurately separate banana bunches from orchards, this study proposes a multi-view extraction framework (MVExt), which alleviates occlusion and interference caused by the complex banana orchard environment by combining multiple views and point cloud projection. BBNeuS combines Signed Distance Field (SDF) supervision with bias consistency, which not only reduces deviations in volumetric rendering but also alleviates viewpoint discrepancies caused by unstable lighting conditions. We conducted segmentation experiments across various scenarios, with all evaluation metrics showing improvement, achieving up to a 12 % increase. In reconstruction experiments, the evaluation scores for PSNR, SSIM, and LPIPS reached 21.17, 0.89, and 0.27, respectively, representing improvements of 21.4 %, 20.3 %, and 20.6 % compared to baseline methods. The MAE metric was well balanced. These results demonstrate that BBNeuS can accurately extract banana-related information and significantly enhance the reconstruction of banana bunches. This research provides valuable insights for 3D phenotypic analysis of banana bunches and holds great significance for the development of intelligent banana orchards.