<i>IoTouch</i>: whole-body tactile sensing technology toward the tele-touch

Abstract

In the era of social distancing, humans tend to interact more with digital edges/devices and robotic avatar. Users mostly interact with cyber-physical systems through audiovisual devices, while tangible interfaces are limited to touch panels. Touch perception not only brings sense of physical contact, texture; but also conveys the intention and emotion of users. Here, we introduce a novel soft tangible device, called IoTouch, as a combination of soft robotics and deep learning, that can act as both robotic skin or haptic sensing interface. The technique aims to implementation of touch sensing with a vision-based approach. We showcase on barrel shape (ellipsoid with two flattened ends) of the IoTouch, where a dataset of tactile images was collected through an autonomous process, in order to train a network for recognition of physical contact states (contact location, contact depth). A Finite Element Method model of the barrel skin was also setup for data acquisition of skin's deformation. The proposed method can detect multiple contacts simultaneously on the entire area of the barrel skin. Obtained result promises a platform for creation of robotic avatars that can sense diverse states of physical touch, through which the users' experience and communication of haptics can be enhanced.