Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


case1246922202.pdf (5.35 MB)

ETD Abstract Container

Abstract Header

A Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm

Thomas, Philip S.
http://rave.ohiolink.edu/etdc/view?acc_num=case1246922202

Abstract Details

2009, Master of Sciences, Case Western Reserve University, EECS - Computer and Information Sciences.
This thesis demonstrates the feasibility of using reinforcement learning (RL) for functional electrical stimulation (FES) control of a human arm as an improvement over (i) previous closed-loop controllers for upper extremities that are unable to adapt to changing system dynamics and (ii) previous RL controllers that required thousands of arm movements to learn. We describe the relevance of the control task and how it can be applied to help people with spinal cord injuries. We also provide simulations that show previous closed-loop controllers are insufficient. We provide background on possible RL techniques for control, focusing on a continuous actor-critic architecture that uses function approximators for its mappings. We test various function approximators, including Artificial Neural Networks (ANNs) and Locally Weighted Regression (LWR) for this purpose. Next, we introduce a novel function approximator, Incremental Locally Weighted Regression (ILWR), which is particularly suited for use in our RL architecture. We then design, implement, and perform clinically relevant tests using ANNs for the two mappings in the continuous actor-critic. During these trials, unexpected behavior is observed and eventually used to create a hybrid controller (that switches among different learning parameter sets) that can both adapt to changes in arm dynamics in 200 to 300 arm movements and remain stable in the long-term. A non-switching controller with similar performance is achieved using ILWR in place of an ANN for the controller's critic mapping.
Michael Branicky, PhD (Advisor)
Antonie van den Bogert, PhD (Committee Member)
Soumya Ray, PhD (Committee Member)
205 p.
Computer Science
reinforcement learning; locally wieghted regression; incremental locally weighted regression; LWR; ILWR; ANN; actor-critic; continuous actor-critic; functional electrical stimulation; FES; adaptive control

Recommended Citations

Citations

  • Thomas, P. S. (2009). A Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm [Master's thesis, Case Western Reserve University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=case1246922202

    APA Style (7th edition)

  • Thomas, Philip. A Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm. 2009. Case Western Reserve University, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=case1246922202.

    MLA Style (8th edition)

  • Thomas, Philip. "A Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm." Master's thesis, Case Western Reserve University, 2009. http://rave.ohiolink.edu/etdc/view?acc_num=case1246922202

    Chicago Manual of Style (17th edition)

case1246922202
806
© 2009, all rights reserved.
This open access ETD is published by Case Western Reserve University School of Graduate Studies and OhioLINK.