Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


devin_schwab_masters_thesis.pdf (1.71 MB)

ETD Abstract Container

Abstract Header

Hierarchical Sampling for Least-Squares Policy Iteration

Schwab, Devin
http://orcid.org/0000-0003-0172-9744
http://rave.ohiolink.edu/etdc/view?acc_num=case1441374844

Abstract Details

2016, Master of Sciences, Case Western Reserve University, EECS - Computer and Information Sciences.
For large Sequential Decision Making tasks, an agent may need to make lots of exploratory interactions within the environment in order to learn the optimal policy. Large amounts of exploration can be costly in terms of computation, time for interactions, and physical resources. This thesis studies approaches to incorporate prior knowledge to reduce the amount of exploration. Specifically, I propose an approach that uses a hierarchical decomposition of the Markov Decision Process to guide an agent's sampling process, in which the hierarchy is treated as a set of constraints on the sampling process. I show theoretically that, in terms of distributions of state-action pairs sampled with respect to hierarchical states, variants of my approach have good convergence properties. Next, I perform an extensive empirical validation of my approach by comparing my methods to baselines which do not use the prior information during the sampling process. I show that using my approach, not only will irrelevant state-action pairs be avoided while sampling, but that the agent can learn a hierarchically optimal policy with far fewer samples than the baseline techniques.
Soumya Ray (Advisor)
Cenk Cavusoglu (Committee Member)
Michael Lewicki (Committee Member)
Harold Connamacher (Committee Member)
117 p.
Computer Science
reinforcement learning; MaxQ; LSPI; Least-Squares Policy Iteration

Recommended Citations

Citations

  • Schwab, D. (2016). Hierarchical Sampling for Least-Squares Policy Iteration [Master's thesis, Case Western Reserve University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=case1441374844

    APA Style (7th edition)

  • Schwab, Devin. Hierarchical Sampling for Least-Squares Policy Iteration. 2016. Case Western Reserve University, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=case1441374844.

    MLA Style (8th edition)

  • Schwab, Devin. "Hierarchical Sampling for Least-Squares Policy Iteration." Master's thesis, Case Western Reserve University, 2016. http://rave.ohiolink.edu/etdc/view?acc_num=case1441374844

    Chicago Manual of Style (17th edition)

case1441374844
461
© 2015, some rights reserved.Creative Commons License Hierarchical Sampling for Least-Squares Policy Iteration by Devin Schwab is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. Based on a work at etd.ohiolink.edu.
This open access ETD is published by Case Western Reserve University School of Graduate Studies and OhioLINK.