A Mixed Reality System for Human Teleoperation in Tele-Ultrasound

Abstract

Many applications including telemedicine, manufactur- ing, and maintenance profit from remote guidance. Ex- isting approaches to tele-ultrasound (US) include robotic teleoperation as well as multimedia applications that combine verbal and graphical guidance. Robotic US systems can provide high precision, low latency, and haptic feedback [1][2][3]. One system has demonstrated clinical utility in trials [4], and much recent work has focused on autonomous robotic US [5]. However, the issues of safe human-robot interaction and guaranteed robust autonomy remain difficult, especially from a reg- ulatory perspective. Further limitations include restricted workspaces, time consuming set-up, large physical size that prevents use in ambulances, and cost, especially compared to inexpensive US systems. Conversely, systems sold by Clarius Mobile Health Corp. and Butterfly Network use a wireless US probe with images and video conferencing available via a cloud interface on a mobile phone application. Though inexpensive and flexible, the desired probe pose and force are given verbally or with some overlays of arrows or pointers on the US image, which is very inefficient, leading to high latency and low precision. We present a novel concept of “Human Teleoperation" through mixed reality which bridges the gap between these two methods. In this control framework, the human follower is controlled as a flexible, cognitive robot such that both the input and the actuation are carried out by people, but with near robot-like latency and precision. This allows teleguidance that is more precise, intuitive, and low latency than verbal guidance, yet more flexible, inexpensive, and accessible than robotic teleoperation. This short paper summarizes the concepts and intro- duces a new design of the communication system to use a secure, high-speed, network-agnostic WebRTC interface. More details and results can be found in [6].