OmniSense: A Visuotactile Sensor for Force, Texture, and Temperature

摘要

nterest in visuotactile sensors has amassed relevance in robotics, haptics, and computer vision due to their ability to perceive high-resolution tactile information and contact measurements [1,3]. One popular visuotactile sensor is the GelSight sensor which comprises a transparent elastomer with a coating on one side, a camera, a set of LED lights, and a transparent support plate [2-4]. The transparent elastomer with reflective coating serves as the contact medium for tactile sensing, which deforms respective to the object's surface geometry upon contact, and the image is captured by a camera [3,5]. From the captured tactile image, the object in contact can be detected and reconstructed in 3D, and corresponding contact forces such as slip torque and shear can be measured [1-5]. These attributes make visuotactile sensors superior to conventional scanners, tactile, and force sensors in various applications [4-6]. medical applications. Compared to human touch, the sensor proved to have greater sensitivity in detecting lumps with the same lump size or depth. From these attributes, the sensor has great use case for external tumor screening and advantageous in performing telemedicine Aside from industrial applications, it has exemplified potential in and in some cases minimally invasive surgeries [7]. While the GelSight sensor is revered for its qualities, it requires two different elastomers to perform the mentioned applications, one with markers and one without markers [3,8]. In 2022, Abad et al. developed a new visuotactile sensor named HaptiTemp and proposed a way to allow force measurements and detections on one elastomer [8]. Instead of using black permanent markers, Abad et al. used Ultraviolet markers that become visible under UV LED illumination, enabling force assessments without being an obstruction during recognition. Accordingly, it is worth mentioning that this research opened an additional use case for visuotactile sensors. From its name, HaptiTemp, the sensor can also measure the temperature of objects in contact, which is achieved using thermochromic pigments with different thresholds layered along the elastomer. However, it was not able to implement metrology applications and 3D reconstruction. Hence, by incorporating multiple thermochromic coatings, high-detail retrographic image production is reduced due to color noise and the elastomer's sensitivity to deform due to the thicker contact coating. Following these researches, we developed a new visuotactile sensor,named OmniSense, that incorporates all the sensors' functionalities to completely sense the trifecta of tactile information - force, texture, and temperature using one sensor. This paper focuses on the hardware and elastomer development of the new sensor.