Remote safety system for a robot tractor using a monocular camera and a YOLO-based method

Abstract

Japanese agriculture faces serious problems caused by labor shortages due to both a decline in the agricultural population and the aging of the remaining agricultural population. Japanese government is actively promoting the use of agricultural robots to address labor shortages issues. According to the Safety Assurance Guidelines for Agricultural Machinery Autonomous Navigation issued by the Ministry of Agriculture, Forestry, and Fisheries of Japan. Human monitoring is necessary during the work of agricultural robots. A remote safety system was developed to support the monitoring of a multi-robot tractor. It uses a monocular camera as an image-input device and a deep-learning method YOLO model as a detector for humans and tractors. The system acquires image data at the robot's local end and sends it to the remote end for an image analysis; it then calculates the relative position of the detected target to the robot tractor. The Q-Q plot and t-test were employed to enhance the accuracy of human positioning. The safety results of the system's analysis are sent back to the tractor for execution and to generate an alert to the human monitor. Human positioning with a relative error of 2.6% at 15 m was obtained. Every target was correctly detected in the 2022–2023 field experiment. These results demonstrate that the remote safety system can support the human monitoring of a robot tractor.