Reinforcement Learning System Automatically Trains Prosthetic Legs

Powered leg prostheses can give amputees the ability to walk for long periods of time and to do so briskly. The reality is that these devices are pretty clunky and programming them to operate smoothly to produce a natural gait takes hours, and the results are rarely perfect.

Scientists at North Carolina State University, the University of North Carolina, and Arizona State University have now developed an automated tuning system that relies purely on a technique called reinforcement learning to do its job. This allows the patient to simply walk with a new prosthetic on a treadmill while guided by a therapist. The system monitors the intentions of the patient and movement of the prosthetic, and adjusts in real time based on the readings.

It takes about about ten minutes to train a prosthetic leg so that it feels quite natural. This is partially due to the fact that the system adjusts a dozen different parameters, such as joint stiffness, throughout each step during the training process. “We begin by giving a patient a powered prosthetic knee with a randomly selected set of parameters,” said Helen Huang, co-author of the study, the announcement. “We then have the patient begin walking, under controlled circumstances. Data on the device and the patient’s gait are collected via a suite of sensors in the device,” Huang says. “A computer model adapts parameters on the device and compares the patient’s gait to the profile of a normal walking gait in real time. The model can tell which parameter settings improve performance and which settings impair performance. Using reinforcement learning, the computational model can quickly identify the set of parameters that allows the patient to walk normally. Existing approaches, relying on trained clinicians, can take half a day.”

Currently the prosthetic leg and sensors are connected via a fat cable to a computer system, which runs the reinforcement learning algorithm and makes the adjustments. The next steps will involve making this system wireless and able to be used by patients in their normal environments. This should improve the quality of the training and lead to more natural walking gaits.

Study in IEEE Transactions on Cybernetics: Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis…