A Unified Deep Imitation Learning and Control Framework for Robot-Assisted Sonography

Abstract

Medical ultrasound scanning is a challenging dexterous manipulation task for robots, even for experienced sonogra-phers, since it involves motion control and force regulation based on the real-time ultrasound image and patient feedback. In this work, we proposed a novel robot-assisted ultrasound scanning framework, integrating deep multimodal imitation learning and model-based compliant control. We investigated the deep imitation learning model to fuse multimodal information, including the RGB image, force profile, ultrasound image, and proprioceptive information for robot-assisted ultrasound scanning artery. The deep imitation learning module predicts reference motion and force command. We designed a compliant controller in Cartesian space to track reference trajectory and desired force. The generalization capability of the deep multimodal imitation learning module and control performance and the quality of the acquired ultrasound image through Phantom. The results show that the proposed approach is able to improve the success rate of procedure completion, and the complete time is reduced.