The Development of an Augmented Reality Gesture Control Human-robot Interface

Abstract

There is an increased need for intuitive methods to control and interact with collaborative robot systems that are driven by industries such as manufacturing that involve complex robot operations. Traditional control and programming approaches that involve handheld devices can be both cumbersome and potentially hazardous for operators, as unexpected movements or malfunctions may result in dangerous situations, they also restrict operators’ movements and hinder their ability to respond quickly to changing situations, ultimately slowing down overall operations. The implementation of cutting-edge interface technology such as Augmented Reality (AR) and gesture control can revolutionise robot systems and propel companies towards Industry 4.0. However, a significant gap exists in the realm of user-friendly and dependable AR-based interfaces that seamlessly integrate with robot systems, guarantee safe and precise operations, and reduce the likelihood of operator errors and accidents.This paper demonstrates the benefits of developing and deploying AR gesture interfaces to empower robotics operators to control and interact with manipulators in more natural and efficient manners. This interface could represent a substantial advancement in addressing the problems presented by conventional control methods, ushering in a new era of robot system control and interaction in complex industrial settings.An experimental approach was developed to investigate the feasibility and effectiveness of using AR to control robot systems. A comparative experiment to evaluate the effectiveness and usability of an Augmented Reality (AR) gesture interface, developed using HoloLens 2 and UR16e robot in contrast to the conventional teach pendant control method. The study aims to provide valuable insights into the utility and user-friendliness of the AR gesture interface for robot system control from users’ perspectives. The devices have been compared using participants who have engaged in a series of tasks involving robot movement, manipulation, and interaction.