High-throughput 3D reconstruction of plants and its application to plant feature segmentation

摘要

This study explores the development and evaluation of high-throughput 3D plant reconstruction and 2D feature detection and segmentation methods for plant phenotyping. A robotic system was employed to collect datasets of individual cucumber plants, utilizing automated mechanisms for efficient, high-throughput data acquisition. Three types of 3D reconstruction methods called Instant-NGP, Nerfacto, and 3D Gaussian Splatting were adopted and compared in terms of rendering quality and speed.Among them, 3D Gaussian Splatting performed the best, achieving PSNR: 25, SSIM: 0.84, LPIPS: 0.20, and also an impressive rendering speed of 6.39 FPS. Novel viewpoint renderings and depth maps further demonstrated its ability to generate accurate and photo-realistic representations of plants. Additionally, rendered images were utilized for training YOLO models to segment plant features into two classes: leaf and fruit. The YOLOv11s model achieved the highest F1 Score (0.932), balancing speed and accuracy. Ultra-view renderings and segmentation provided valuable insights into plant morphology, including leaf and fruit counts, paving the way for scalable, automated phenotyping applications. This study highlights the potential of integrating 3D Gaussian Splatting with advanced segmentation models for precise and efficient plant phenotyping. • Introducing a throughput system for data collection tailored for subsequent 3D reconstruction of cucumber plants. • Implementation of Neural Radiance Fields and 3D Gaussian Splatting techniques to create 3D models of cucumber canopies. • Utilization of 2D instance segmentation to detect plant features on images rendered from the 3D reconstruction models.