A Real-Time Human Pose Measurement System for Human-In-The-Loop Dynamic Simulators

摘要

The possibility of achieving real-time evaluation of human pose enables the ability to control robotic platforms based on user’s pose (or even on user’s inertial properties) in Human-In-The-Loop simulators, for sports and rehabilitation as an example. This study presents a vision-based, marker-less measurement system for real-time 3D human pose estimation. The system exploits pre-trained 2D human pose detection models and integrates an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha \text {-}\beta \text {-}\gamma $ </tex-math></inline-formula> filter to reduce fluctuations in detected key points. It also introduces a novel Weighted Direct Linear Triangulation method, enhancing 3D reconstruction accuracy by assigning higher weights to key points consistent across current and previous frames. The method’s accuracy and execution time are assessed using the public Human3.6M dataset, evaluating different model configurations, formats, camera setups, and acquisition modes for real-time applications. The YOLOv8x-pose model with four cameras achieves the highest accuracy, with a Mean-Per-Joint Position Error of 18.2 mm and an execution time of 15 ms, outperforming state-of-the-art methods. Converting models to the TensorRT framework reduces execution time by 4.2 ms without significant accuracy loss. The system is integrated into a clinical rehabilitation device, a three-degree-of-freedom parallel kinematic machine, to facilitate patient participation in exergames. The proposed human-pose estimation method achieves real-time performance, enabling the motion platform to be controlled dynamically based on the patient’s actual standing pose.