Vision‐Assisted Avocado Harvesting with Aerial Bimanual Manipulation

Abstract

Robotic fruit harvesting holds potential in precision agriculture to improve harvesting efficiency. While ground mobile robots are mostly employed in fruit harvesting, certain crops, like avocado trees, cannot be harvested efficiently from the ground alone. This is because of unstructured ground and planting arrangement and high‐to‐reach fruits. In such cases, aerial robots integrated with manipulation capabilities can pave new ways in robotic harvesting. This paper outlines the design and implementation of a bimanual aerial robot that employs visual perception and learning to detect avocados, reach, and harvest them autonomously. The dual‐arm system comprises a gripper and a fixer arm, to address a key challenge when harvesting avocados: once grasped, applying a rotational motion is the most mechanically efficient way to detach the avocado from the peduncle; however, the peduncle may store elastic energy preventing the avocado from being harvested. The fixer arm aims to stabilize the peduncle, allowing the gripper arm to harvest. The integrated visual perception process enables the detection of avocados and the determination of their pose; the latter is then used to determine target points for a bimanual manipulation planner. Several experiments are conducted in controlled indoor and outdoor settings to assess the efficacy of each component individually. Further, an integrated experiment in outdoor semicontrolled settings is used for feasibility assessment of the overall system. Results demonstrate that all different components can work synergistically to enable robotic avocado harvesting in (semi‐)controlled settings. Results also highlight limitations of an airborne harvesting solution and reveal tradeoffs to be considered in the selection of a harvesting robot.