Tracking of human operator arms oriented to the control of two robotic arms

Abstract

This article presents the development of machine vision algorithms that allows to manipulate two robotic arms by the operator, replicating the movements of his own arms, accomplishing control of four degrees of freedom per arm. The development is aimed to provide an environment of natural, wireless and non-invasive control, providing security to the operator in the face of achievement of unsafe tasks, either for removal of hazardous, explosive or radioactive materials, as well as possibly toxic environments or just remote control from a remote site. The tracking algorithm of the two arms is based on their detection using the skeleton tool of Microsoft Kinect sensor, as well as image processing algorithms like color space conversion, segmentation and thresholding to control the gripper arm. It was possible to obtain real-time control of the arms to four degrees of freedom, according to the monitoring of the arms of the operator in the ranges of sensor coverage.