Muscle-Joint Feature Fusion for Swimming Pattern Recognition With 1D-CNN Classifier

Abstract

Swimming pattern recognition plays an important role in underwater wearable robot control and competitive sports training. However, serious disturbances of harsh underwater environments result in few available sensing methods, and existing single-mode sensing inherently limits the efficiency of swimming monitoring. This article combines muscle deformations measured by flexible capacitive sensors with limb joint rotations by inertial measurement units (IMUs). Heatmaps and polar coordinate systems are used to exhibit differences of cyclic features in four strokes. The t-SNE method is used to convert high-dimensional fusion features to a 2-D plane for visualization. Using a 1-D convolutional neural network (1D-CNN) classifier for swimming pattern recognition at three kicking frequencies, the effectiveness of the method was experimentally validated by nine subjects with an average accuracy of 97.9%, while that in single sensing modality is 82.6% with muscle deformations and 85.9% with limb rotations. To the best of our knowledge, this article is the first to use multimodal fusion for swimming motion monitoring and is expected to enhance underwater wearable sensing devices for human-centered robotics and sports science.