A YOLOv7-based robotic harvesting system for Agaricus bisporus using a depth camera

摘要

To achieve accurate detection of button mushrooms (Agaricus bisporus), a harvesting method based on the Robot Operating System (ROS) is proposed. This method combines a convolutional neural network named YOLO (You Only Look Once), a depth camera, and a robotic arm working in coordination. The system integrates advanced image recognition capabilities with the robotic arm to improve harvesting efficiency and accuracy. To enhance the model’s detection accuracy while considering the lightweight requirements for mobile deployment, several modules were incorporated into the You Only Look Once (YOLO)v7 network. Experimental results showed that the improved YOLOv7 network increased the average detection accuracy by 1.9% over the original YOLOv7 network, with a significant improvement of 5.24% in [email protected]:0.95. Additionally, the parameter count and computational complexity of the improved YOLOv7 model were significantly reduced. The parameter count decreased by 5.2M, making the model more lightweight and reducing memory use and computational resource demands. The computational complexity of the improved model decreased to 82.1 GFLOPS, optimized from the original model’s 105.1 GFLOPS, resulting in higher processing efficiency and making it suitable for embedded systems or real-time applications. The improved model was tested on a button mushroom harvesting system and automated harvesting experiments were conducted. The results showed that the model accurately detected button mushrooms and successfully performed mushroom harvesting, achieving a detection speed of 70.9 frames per second and a harvesting success rate of 80%. This demonstrates the potential of replacing labor-intensive manual harvesting with automated solutions, providing a feasible approach for the future development of agricultural mechanization.