Real-time detection of broccoli crops in 3D point clouds for autonomous robotic harvesting

Abstract

Real-time 3D perception of the environment is crucial for the adoption and deployment of reliable autonomous harvesting robots in agriculture. Using data collected with RGB-D cameras under farm field conditions, we present two methods for processing 3D data that reliably detect mature broccoli heads. The proposed systems are efficient and enable real-time detection on depth data of broccoli crops using the organised structure of the point clouds delivered by a depth sensor. The systems are tested with datasets of two broccoli varieties collected in planted fields from two different countries. Our evaluation shows the new methods outperform state-of-the-art approaches for broccoli detection based on both 2D vision-based segmentation techniques and depth clustering using the Euclidean proximity of neighbouring points. The results show the systems are capable of accurately detecting the 3D locations of broccoli heads relative to the vehicle at high frame rates.