Automated Harvesting of Green Chile Peppers with a Deep Learning-based Vision-enabled Robotic Arm

Abstract

The green chile crop is entirely hand-harvested in New Mexico while the growing labor shortage has caused a significant reduction in production. This work presents the robotic harvesting of chile peppers in a lab setting, employing a 6-DOF robotic arm with a scissor-type cutting end-effector. The system utilizes a machine learning-based computer vision and a depth camera to detect and localize chile peppers in the camera frame. The locations are then transformed into the robot’s operational frame. A motion planning algorithm was developed to minimize the robot’s travel time for harvesting. A correction equation is derived to address inaccuracies in camera-based localization while eliminating chiles that are not reachable for the robot. From a dataset of 86 chile peppers, the study reports key harvesting metrics: a detection success rate of 62.8%, a localization success rate of 90.74%, a detachment success rate of 55.10%, a harvest success rate of 31.39%, and a damage rate of 6.97%.