Classification Method of Visual-Tactile Fusion Dataset Based on CNN-TCN

Abstract

Aiming at the problem that it is difficult to identify similar objects using visual information alone in a space manipulator operating environment with complex light conditions, a dataset for similar object classification is proposed and a CNN-TCN structure is used for visual-tactile feature fusion research. The visual-tactile dataset consists of images and tactile force sequences, and is collected through a data acquisition system consisting of a robotic arm, a two-finger gripper, a camera and a pair of tactile sensors. A feature extraction network combining CNN and TCN is proposed to respectively extract the spatial features of visual data and the temporal and spatial features of tactile data, and use this dual-modal information to improve the accuracy of similar object recognition.