Skip to Main Content
 

Global Search Box

 
 
 
 

ETD Abstract Container

Abstract Header

STUDY ON INFORMATION THEORY: CONNECTION TO CONTROL THEORY, APPROACH AND ANALYSIS FOR COMPUTATION

Theeranaew, Wanchat

Abstract Details

2015, Doctor of Philosophy, Case Western Reserve University, EECS - System and Control Engineering.
This thesis consists of various studies in information theory, including its connection with control theory and the computational aspects of information measures. The first part of the research investigates the connection between control theory and information theory. This part extends previous results that mainly focused on this connection in the context of state estimation and feedback control. For linear systems, mutual information, along with the concepts of controllability and observability, is used to derive a tight connection between control theory and information theory. For nonlinear systems, a weaker statement of this connection is established. Some explicit calculations for linear systems and interesting observations about these calculations are presented. The second part investigates the computation of mutual information. An innovative method to compute the mutual information between two collections of time series data based on a Hidden Markov Model (HMM) is proposed. For continuous-valued data, a HMM with Gaussian emission is used to estimate the underlying dynamics of the original data. Mutual information is computed based on the approximate dynamics provided by the HMM. This work improves the estimation of the upper and lower bounds of entropy for Gaussian mixtures, which is one of the key components in this proposed method. This improvement of these bounds are shown to be robust compared to existing methods in all of the synthetic data experiments conducted. In addition, this research includes the study of the computation of Shannon mutual information in which the strong assumptions of independence and identical distribution (i.i.d.) are imposed. This research shows that even if this assumption is violated, the results process a meaningful interpretation. The study of the computation of Shannon mutual information for continuous-valued random variables is included in this research. Three coupled chaotic systems are used as exemplars to show that the computation of normalized mutual information is relatively insensitive to the number of quantized states, although quantization resolution does significantly affect the unnormalized mutual information. The same coupled chaotic systems are used to show that the quantization method also does not significantly affect the normalized mutual information. Simulations from these chaotic systems also show that normalized Shannon mutual information can be used to detect the different (fixed) coupling strengths between two subsystems. Two modified information measures, which enforce sensitivity to time permutation, are compared on these three systems. By using piecewise constant coupling and monotonically decaying coupling, the simulation results show that normalized mutual information can track time-varying changes in coupling strength for these chaos systems to a certain degree.
Kenneth Loparo (Advisor)
Vira Chankong (Committee Member)
Marc Buchner (Committee Member)
Richard Kolacinski (Committee Member)
82 p.

Recommended Citations

Citations

  • Theeranaew, W. (2015). STUDY ON INFORMATION THEORY: CONNECTION TO CONTROL THEORY, APPROACH AND ANALYSIS FOR COMPUTATION [Doctoral dissertation, Case Western Reserve University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=case1416847576

    APA Style (7th edition)

  • Theeranaew, Wanchat. STUDY ON INFORMATION THEORY: CONNECTION TO CONTROL THEORY, APPROACH AND ANALYSIS FOR COMPUTATION. 2015. Case Western Reserve University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=case1416847576.

    MLA Style (8th edition)

  • Theeranaew, Wanchat. "STUDY ON INFORMATION THEORY: CONNECTION TO CONTROL THEORY, APPROACH AND ANALYSIS FOR COMPUTATION." Doctoral dissertation, Case Western Reserve University, 2015. http://rave.ohiolink.edu/etdc/view?acc_num=case1416847576

    Chicago Manual of Style (17th edition)