Improving Safeguards and Functionality in Industrial Collaborative Robot HMIs through GUI Automation

Abstract

Due to safety and security concerns, industrial human-machine interfaces (HMIs) cannot be updated as frequently and easily as other software. This leads to an innovation gap in productive industrial environments, which is aggravated by the demand that innovation yield immediate gains in productivity and profitability. Hence, industrial HMIs often lag behind the state of the art, which may pose risks in terms of the safety and security of human-machine interactive systems, such as collaborative robots, while impeding their interoperability with machine vision and other novel technologies. In response, this paper explores the potential and limits of novel tools and techniques from the domain of graphical user interface (GUI) automation, which can be used to extend existing robot HMIs in a flexible and pragmatic way. In this sense, the paper introduces a novel architecture for HMI automation and evaluates it using five non-exhaustive use cases for collaborative robots. Our evaluation demonstrates the effectiveness of the approach in various industrial application contexts that require specific extensions to off-the-shelf robotic software.