Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


C.Hou Dissertation.pdf (929.33 KB)

ETD Abstract Container

Abstract Header

Dynamic Programming under Parametric Uncertainty with Applications in Cyber Security and Project Management

Hou, Chengjun
http://orcid.org/0000-0002-0351-5179
http://rave.ohiolink.edu/etdc/view?acc_num=osu1437676379

Abstract Details

2015, Doctor of Philosophy, Ohio State University, Industrial and Systems Engineering.
The trustworthiness of models and optimization is limited because the associated systems might be changing and data about them can be limited, i.e., there is "parametric" uncertainty. This dissertation provides applications and theory related to mitigating the effects of changing systems and data limitations in optimal decision-making. The primary application considered relates to reducing the maintenance costs associated with cyber security. By selecting optimal policies addressing data limitations, losses from stolen information and maintenance costs can be balanced. The approximated expected savings from implementing the suggested policies at a large Midwestern organization is over $14M with a discount factor of 0.95 monthly. The dissertation also integrates data and dynamic programming models for project management decision-making that accounts for coordination and planning costs. This facilitates more accurate schedules with significant cost savings. Insights are provided into the choice between traditional planning methods and agile project management methods that reduce planning complexity. In many situations, we find that the so-called optimal approaches are suboptimal because they fail to address sizable coordination and planning costs. Two types of parametric uncertainty are explored here, each of which results in fundamentally different formulations and solution schemes. The first type of uncertainty considered relates to system parameters fluctuating over time randomly. The related models differ from ordinary inhomogeneous approaches because the specific parameters are not known and are assumed to fluctuate with known distributions. Associated decision problems are referred to as "Markov decision processes with random inhomogeneity" and proposed optimal solutions methods. Proof is given that the solution produced by backward induction is optimal for the finite horizon problems, and that the value-iteration-based algorithm gives solutions converging to the infinite horizon solutions, together with results regarding monotonicity property and rate of the convergence. The second type of parametric uncertainty is caused by insufficient data for parameter estimation, i.e., "data-driven" uncertainty. Previous researchers studying data-driven Markov decision processes declare the problem is intractable. Therefore, they propose approximation methods. We prove that their methods can approximate suboptimal solutions by a numerical example. We also provide a dynamic programming algorithm to generate data-driven optimal policies with learning. We do this by demonstrating that the problem is equivalent to partially observable Markov decision processes. Further, by exploiting the structure of the problem and bounds assuming perfect information, we develop a bounding heuristic method for the infinite horizon problems.
Theodore Allen (Advisor)
Nicholas Hall (Committee Member)
Gagan Agrawal (Committee Member)
146 p.
Industrial Engineering; Operations Research
Cyber security; Dynamic programming; Markov decision processes; Parametric uncertainty; Project management

Recommended Citations

Citations

  • Hou, C. (2015). Dynamic Programming under Parametric Uncertainty with Applications in Cyber Security and Project Management [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1437676379

    APA Style (7th edition)

  • Hou, Chengjun. Dynamic Programming under Parametric Uncertainty with Applications in Cyber Security and Project Management. 2015. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1437676379.

    MLA Style (8th edition)

  • Hou, Chengjun. "Dynamic Programming under Parametric Uncertainty with Applications in Cyber Security and Project Management." Doctoral dissertation, Ohio State University, 2015. http://rave.ohiolink.edu/etdc/view?acc_num=osu1437676379

    Chicago Manual of Style (17th edition)

osu1437676379
1,129
© 2015, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.