Towards AI-Controlled Movement Restoration: Learning FES-Cycling Stimulation with Reinforcement Learning

摘要

Functional electrical stimulation (FES) has been increasingly integrated with other rehabilitation devices, including rehabilitation robots. FES cycling is one of the common FES applications in rehabilitation, which is performed by stimulating leg muscles in a certain pattern. The appropriate pattern varies across individuals and requires manual tuning which can be time-consuming and challenging for the individual user. Here, we present an AI-based method for finding the patterns, which requires no extra hardware or sensors. Our method starts with finding model-based patterns using reinforcement learning (RL) and customised cycling models. Next, our method fine-tunes the pattern using real cycling data and offline RL. We test our method both in simulation and experimentally on a stationary tricycle. Our method can robustly deliver model-based patterns for different cycling configurations. In the experimental evaluation, the model-based pattern can induce higher cycling speed than an EMG-based pattern. And by using just 100 seconds of cycling data, our method can deliver a fine-tuned pattern with better cycling performance. Beyond FES cycling, this work is a case study, displaying the feasibility and potential of human-in-the-loop AI in real-world rehabilitation.