A Neural Network-Based Lower Extremity Joint Angle Estimation from Insole Data

摘要

Human pose estimation is among the emerging fields which are being actively researched and attracting interest for various purposes. Applications could range from healthcare to robotics, from human augmentation to virtual reality. Especially, in aging societies like Japan, it could be a great help in monitoring the status of elders in elderly-care centers to notify and prevent injuries. Existing methods vary in sensor type and sensor quantity and utilized algorithms. Optical motion capture systems and multiple inertial sensor systems have been the popular ones. However, both systems suffer from high maintenance and setup costs, space limitations, and drifting errors. This restricted most existing studies to only indoor activities. In this research, we developed a deep learning-based lower limb joint angle estimator using only two insole sensors placed in each shoe. Insole sensors are convenient for attachment and are less prone to skin artifacts. A 45-minute walking insole and optical data were collected in a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$6\mathrm{m}\times 3\mathrm{m}$</tex> indoor space. As both systems have different sampling frequencies, synchronization was performed in a semi-automatic way. The optical motion capture was used to validate the estimation system. Two neural network algorithms namely convolutional neural network (CNN) and bi-directional long short-term memory (BLSTM) neural network were investigated in this study. After the training, CNN achieved better results with an average mean absolute error (MAE) of 3.17° for four sagittal plane joint angles, these include the extension/flexion of both hip and knee joints. Finally, the trained model was tested using unseen datasets which showed a promising result that can be used as a foundation for further analysis.