Multisensory surgical support system incorporating, tactile, visual and auditory perception modalities

Abstract

The incorporation of novel broad band sensory modalities, integrating tactile technology, with visual and auditory signals into the evolution of the next generation of surgical robotic is likely to significantly enhance their utility and safety. In this paper considerations are made of a system, where tactile information together with visual and audio feedback are integrated into a multisensory surgical support platform. The tactile sensor system uses a piezoelectric transducer (PZT) system to evaluate the haptic properties of tissues. The spatial position of the sensor is tracked by a video camera, visualizing the location of the marker. Tactile information is additionally converted to an audio signal, to represent tissue properties in terms of a frequency/amplitude modulated signal. Representative data were obtained from biological tissues demonstrating that the technology developed has potential applications in virtual systems or robotic tele-medical care. In view of these technical developments, consideration is made as to whether visual audio and tactile modalities act as independent sources of information.