Corn pose estimation using 3D object detection and stereo images

摘要

Corn is an important staple crop. Obtaining the pose and dimensions of corn is key for automating corn cultivation, mainly using robotic arms or similar devices to perform precise operations, such as pesticide spraying, measurement, or precise picking. In this study, the Stereo-Corn-Pose Detection (SCPD) algorithm was proposed, which used three dimensional (3D) object detection to obtain the pose and dimensions of corn with stereo images. This algorithm includes pitch angle detection, which is absent in traditional 3D object detection. The SCPD algorithm consists of two models: the union box detection model, FCOS-Stereo, based on the anchor-free network FCOSNet, and the 3D bounding box regression model, Cross-Stereo-EfficientFormer. This regression model incorporates a cross-attention mechanism into EfficientFormer to extract and fuse features effectively. This work constructed a dataset comprising 2,700 samples for training and 300 samples for testing. In the test set, this work achieved a union bounding box m A P of 85.3%, representing a 3.5% improvement over the original FCOSNet model. It also achieved 88.3% A P 3 D and 85.6% A P B E V for 3D bounding box regression, making increases of 5% and 4.7%, respectively, compared to the traditional 3D object detection method, IDA-3D. The results indicate an accuracy of approximately 91% in detecting corn dimensions and pose. Therefore, the SCPD algorithm offers a novel framework for obtaining the 3D dimensions and pose of corn and promotes precision and smart agriculture for corn cultivation. • The SCPD algorithm has been proposed to detect 3D pose and dimensions of corn. • It includes the FCOS-Stereo and Cross-Stereo-EfficientFormer models. • FCOS-Stereo model achieved superior bounding box detection. • Cross-Stereo-EfficientFormer had better AP3D and APBEV than traditional models. • They are effective for union bounding box detection and 3D bounding box regression.