Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Hao_Li_Dissertation.pdf (2.37 MB)

ETD Abstract Container

Abstract Header

Decentralized Multi-Agent Collision Avoidance and Reinforcement Learning

Li, Hao
http://rave.ohiolink.edu/etdc/view?acc_num=osu1618840664964088

Abstract Details

2021, Doctor of Philosophy, Ohio State University, Electrical and Computer Engineering.
This dissertation studies decentralized multi-agent collision avoidance and reinforcement learning (RL) for Markov Decision Process (MDP) with state-dependent action constraints. The multi-agent collision avoidance problem is a fundamental problem in robotics, and it can be generally defined as multiple robots navigating in a shared environment while avoiding collisions with each other. It is well known in the literature that multi-agent collision avoidance is challenging to solve, mainly due to complex dynamics constraints, limited information for each agent, and strict safety constraints. We first propose a decentralized collision avoidance algorithm for heterogeneous multi-agent systems by introducing the extended control obstacles (ECOs). Pairwise state-dependent action constraints from ECOs are introduced to avoid pairwise collisions, which provides strict safety guarantees for heterogeneous linear systems. The overall collision avoidance algorithm for each agent is formulated as a simple convex optimization, which can be solved in real-time. The proposed approach can handle complicated scenarios with uncontrolled agents, nonlinear agents, and obstacles. In the second part of this dissertation, we propose a fast RL-based decentralized collision avoidance algorithm for general nonlinear agents with continuous action space. To reduce online computation, we first decompose the multi-agent scenario and solve a two agents collision avoidance problem via RL. When extending the trained policy to a multi-agent problem, safety is enforced by introducing state-dependent action constraints from the optimal reciprocal collision avoidance (ORCA). The overall collision avoidance action could be found through a simple convex optimization in real-time. Inspired by the collision avoidance algorithms that incorporate state-dependent action constraints, we study RL for continuous MDPs with and state-dependent action constraints. We establish the convergence of fitted value iteration and fitted Q-value iteration. We further extend the algorithms and the convergence result to the case of monotone MDPs, where a function approximating class for the monotone MDPs is identified.
Abhishek Gupta (Advisor)
Wei Zhang (Advisor)
Parinaz Naghizadeh (Committee Member)
Levent Guvenc (Committee Member)
104 p.
Electrical Engineering

Recommended Citations

Citations

  • Li, H. (2021). Decentralized Multi-Agent Collision Avoidance and Reinforcement Learning [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1618840664964088

    APA Style (7th edition)

  • Li, Hao. Decentralized Multi-Agent Collision Avoidance and Reinforcement Learning. 2021. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1618840664964088.

    MLA Style (8th edition)

  • Li, Hao. "Decentralized Multi-Agent Collision Avoidance and Reinforcement Learning." Doctoral dissertation, Ohio State University, 2021. http://rave.ohiolink.edu/etdc/view?acc_num=osu1618840664964088

    Chicago Manual of Style (17th edition)

osu1618840664964088
395
© 2021, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.