Controlling a robotic arm with Augmented reality

Abstract

Under the circumstances of a patient’s upper limb disability, aided by a robotic arm with faulty controls, assistance is needed, using augmented reality as an auxiliary. Our system, with a headset, using an internet connection and an augmented reality device, placed on the assistant’s head, can ensure communication between the two, for both remote supervision and control. The assistant can enhance the control over the robotic arm, while having a head up display on the augmented reality glasses, based on what the patient sees. The communication is established through PC or mobile devices, connected to the internet. Having the patient’s view, and enhanced control over the robotic arm, the assistant can interact with nearby smart objects.