Real-time human-in-the-loop remote control for a life-size traffic police robot with multiple augmented reality aided display terminals

Abstract

Policing of road traffic is listed in the most hazardous tasks since many traffic police personnel injured in a series of accidents at intersections. A life-size traffic cop robot called “IWI” is invented as an alternative to human traffic police in the street for directing traffic on point duty, and this paper proposes a human-robot cooperation scheme with wearable augmented glasses as an HMI to obtain a more immersing teleoperation experience with the aid of wearable augmented reality glasses. First, a humanoid robot is tailored for mobile surveillance with raspberry pi camera based eyes and omnidirectional Mecanum wheels. Second, the real-time video stream acquired by the on-site robot is distributed to multiple terminals, such as the central sever, the wearable augmented reality glasses and the mobile control tablet. Third, with a depth of focus estimation the augmented indications are displayed to aid to understand the remote scenario. Finally, the human policing results, such as STOP, PULL OVER, TURN-LEFT, are compiled and programmed as simplified patterns to control the robot body/hand pantomime. Experimental results show that the proposed methodology and control scheme is feasible in real-time application with high real-time performance of less than 0.5s latency, and open possibilities of easing the traffic jam via simultaneously scheduling multiple traffic cop robots.