Real-Time Dynamic Gesture Recognition for Human-Robot Collaboration in Rescue Operations

Abstract

This paper provides a methodology for dynamic gesture recognition for enhanced Human-Robot Collaboration in search and rescue missions. This research was conducted at DFKI in collaboration with THW, Germany, where an autonomous robot is built for the assistance of emergency workers during disaster scenarios for transporting the materials. This assistance robot will be highly functional in risk-prone areas where the safety of workers is in danger. A set of standard gestures were predefined and those will be used as commands from humans to the robot. Due to the criticality of the area of application of this assistance robot, the identification and interpretation of the gestures must be in real-time and extremely accurate. In this paper, we compare different methods and train different pipelines capable of performing dynamic gesture recognition. The final model pipeline can understand the difference between static as well as dynamic gestures and recognize both of them accurately. The key challenge to attaining real-time gesture recognition is the models's capability to give rapid and highly accurate responses.