Vision and neural control for an orange harvesting robot

Abstract

We describe the system control architecture of a large orange harvesting robot. This robot has two independent electrically driven telescopic arms mounted on a common platform which is itself held by a large hydraulic arm. This arm, in turn, is mounted on a tracked vehicle. The telescopic arms have cameras within the end-effecters, which are used to detect and measure the position and distance of the fruit within the canopy of a tree. Most of the development and control software was implemented using the matrix-based Virtual Machine Language (VML). This language was designed to implement neural networks, and has been extended and enhanced for robotic applications and the particular low level control requirements of the hardware. The device drivers provide the interface to frame grabbers, motor drivers, digital interface electronics, proximity detectors, and file handling. The same interface is used to implement interprocess communications with display and monitoring tools.