ARHands: A Hand Gesture Dataset with HCI in Augmented Reality Surgery

Abstract

AR surgical environments have gained attention for providing real-time medical imaging and other critical surgical reference information to doctors. Robots can offer essential assistance to doctors in the highly controlled surgical environment. To achieve interaction between surgeons and robots, this paper first constructs a first-person perspective gesture dataset within an AR surgical environment. Next, a gesture recognition algorithm incorporating a color attention module is proposed. Finally, an AR surgical human-machine interaction environment is established. In experiments, the RegNetCA gesture recognition algorithm achieved a 98.28% gesture recognition accuracy. Taking the robotic arm assisting in delivering surgical instruments as an example, the effectiveness of the gesture-based human-machine interaction algorithm was verified.