Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Goldblatt_Thesis.pdf (1.13 MB)

ETD Abstract Container

Abstract Header

Model-Free Reinforcement Learning for Hierarchical OO-MDPs

Goldblatt, John Dallan
http://rave.ohiolink.edu/etdc/view?acc_num=case1642700696875092

Abstract Details

2022, Master of Sciences, Case Western Reserve University, EECS - Computer and Information Sciences.
This thesis studies Object-Oriented Markov Decision Processes (OO-MDPs), which extend MDPs with prior knowledge about the shared dynamics of similar objects in the environment. Existing work presents model-based algorithms that leverage the properties of OO-MDPs and adhere to the Knows What It Knows (KWIK) framework. In practice, models may not be easy to estimate and the KWIK framework may still lead to slow performance in a reinforcement learning context. In this thesis, I first introduce a new model-free learning algorithm for OO-MDPs based on Q-Learning. Though my approach is not KWIK, I show empirically that it exhibits significantly faster convergence than the KWIK and flat baselines. Next, I extend hierarchical reinforcement learning (HRL) to use OO-MDPs in the same manner. HRL uses a task hierarchy as prior information to reduce the overall problem into a set of smaller tasks. I show that HRL and OO-MDPs have a natural synergy, and I propose a novel model-free OO-HRL algorithm. I show empirically that this algorithm has better sample complexity than either HRL or OO-MDP algorithms alone.
Soumya Ray (Advisor)
Michael Lewicki (Committee Member)
Harold Connamacher (Committee Member)
M. Cenk Cavusoglu (Committee Member)
103 p.
Artificial Intelligence; Computer Science
reinforcement learning; RL; hierarchical reinforcement learning; HRL; hierarchical; hierarchy; object-oriented reinforcement learning; object-oriented; OO-MDP; OOMDP; MDP; Q-Learning; QLearning; MaxQ; KOOL

Recommended Citations

Citations

  • Goldblatt, J. D. (2022). Model-Free Reinforcement Learning for Hierarchical OO-MDPs [Master's thesis, Case Western Reserve University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=case1642700696875092

    APA Style (7th edition)

  • Goldblatt, John. Model-Free Reinforcement Learning for Hierarchical OO-MDPs. 2022. Case Western Reserve University, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=case1642700696875092.

    MLA Style (8th edition)

  • Goldblatt, John. "Model-Free Reinforcement Learning for Hierarchical OO-MDPs." Master's thesis, Case Western Reserve University, 2022. http://rave.ohiolink.edu/etdc/view?acc_num=case1642700696875092

    Chicago Manual of Style (17th edition)

case1642700696875092
42
© 2022, all rights reserved.
This open access ETD is published by Case Western Reserve University School of Graduate Studies and OhioLINK.