Two-Stage Super-Resolution for Classical 3-D Tactile Sensor Arrays With DT-Driven Enhancement

Abstract

While 3-D tactile sensor arrays featuring traditional 3-D measurement structures have been extensively studied, their limited spatial resolution hinders their ability to perceive small objects during dexterous manipulation. Moreover, existing works on tactile super-resolution (SR) still suffer from lack of contour perception, mapping accuracy, and difficult access to SR datasets. Therefore, a prominent two-stage SR algorithm with digital-twin (DT)-driven enhancement is proposed for sensor arrays with classical 3-D measurement structures (C3DMS). First, based on the intrinsic features of C3DMS, the DT-driven observation of low-/high-resolution tactile sensor arrays is introduced for reliable and accurate tactile SR datasets. Second, a novel deconstructive interpolative upsampling (DIU) is then proposed for DT-driven enhancement and SR multiplier augmentation. Furtherly, an interpolation-convolution two-stage SR network is designed, where the DIU is proposed as the first-stage network to effectively and accurately increase the scale of tactile information, and a convolutional neural network with channel features learning is further constructed as the second stage for high-quality quadruple SR. Comprehensive experiments demonstrate the reliability of the DT dataset and the effectiveness of our approach, with the DT-trained network achieving quadruple SR (PSNR:31.12 and SSIM:0.965) for the self-made sensor, enabling clear recognition of complex Braille letters and higher positioning resolution (0.625 mm) for small objects (e.g., connector of RF antennas). Our SR tactile sensing framework realizes the assembly of RF antennas and holds promise for enhancing the dexterity of robotic manipulation.