With the progress in scientific research and practical application, mathematical models are being improved continuously to explain the nature and advance the technology. This dissertation involves both theoretical and empirical studies on mathematical modeling. From a theoretical standpoint, it investigates model selection and analysis; from an empirical perspective, it explores channel and source coding.
The question of how to decide among competing explanations of data is at the heart of the scientific enterprise. Choosing competing models based solely on the goodness of fit can result in the selection of an unnecessarily complex model that overfits the data. The dilemma is how to compromise both goodness of fit and model complexity. Among various model selection criteria, the Minimum Description Length (MDL) principle is a relatively recent method for inductive inference, which embodies the principle of Occam's razor. In applying MDL to the selection of parametric models, one of the main obstacles is to calculate Fisher information. This study presents a general formula to compute Fisher information with multinomial or normal distribution for any mathematical model.
Another focus of the current research is on componential analysis, which investigates how and how much each parameter affects mathematical model's ability to fit arbitrary patterns of data. To assess the relative importance of each parameter for such an ability is critical to both model selection and model building. The goal of the research along this venue is to establish a unified theory, under which complex modeling procedures can be analyzed in terms of the contribution of each parameter.
Essentially, coding is the direct implementation of mathematical modeling. Channel coding and source coding are the practical applications of two important concepts in the information theory: channel capacity and entropy. This study examines these concepts in two particular cases respectively: bandwidth efficient nonsystematic turbo codes and bitmap index compression through an integrated reorganization.
The investigation starts with an introduction on mathematical modeling and the overview of the study. It is then organized by four consecutive themes in the following chapters: MDL model selection, componential analysis, turbo codes, and bitmap compression.