Enhancing robotic unstructured bin-picking performance by enabling remote human interventions in challenging perception scenarios

Abstract

We present an approach that enables a robot to initiate a call to a remote human operator and ask help in resolving automated perception system failures during bin-picking operations. Our approach allows a robot to evaluate the quality of part recognition and pose estimation, based on a confidence-measure, and thereby determine whether to proceed with the task execution or to request assistance from a human in resolving the predicted perception failure. We present an automated perception algorithm that performs the joint task of part recognition and 6 degree-of-freedom pose estimation, and has built-in features to initiate the call to the human when needed. We also present the underlying mechanism for a rationalized basis for making the call to the human. If uncertainty in part detection leads to perception failure, then human intervention is invoked. We present a new user interface that enables remote human interventions when necessary. We report results from experiments with a dual-armed Baxter robot to validate our approach.