An important but challenging perspective of cognitive modeling is the generalization of a model. That is, whether a cognitive model can account for data observed under alternative experimental settings, either with new participants, under new experimental designs, or in different experimental tasks tapping the same cognitive processes. Most extant generalizability measures for model evaluation focus on the generalization of a model to a new participant sample, but within a given experimental setting. The present dissertation project develops a model evaluation method, dubbed as the hierarchical generalization models, to expand the current generalizability measures and assess the between-experiment generalization. This method utilizes the hierarchical Bayesian modeling approach to account for multilevel data combined from different experimental settings, to separate various sources of variability, and to identify generalizable model assumptions that are related to underlying cognitive processes.
This dissertation examines the applicability and plausibility of the hierarchical generalization modeling framework in the context of studies of behavioral decision-making. Two major experimental paradigms, the decision-from-description and decision-from-experience experiments, are extensively discussed with regard to the modeling of experimental data and the theoretical implication on the generalization of decision-making processes. The hierarchical generalization modeling framework demonstrates its suitability for these decision-making paradigms through simulation studies and secondary data analyses.
Simulation studies in Study 1 and Study 2 demonstrate how to develop hierarchical generalization models to simultaneously model the experimental data from multiple decision-making paradigms. It also shows that hierarchical generalization models can appropriately recover the model structures of Cumulative Prospect Theory based models, which have been one of the mainstream theories in decision-making. Secondary data analyses in Study 3 and Study 4 further extend the application of hierarchical generalization models to heuristic-based stochastic models, and demonstrate the application in an empirical data set. The use of the hierarchical generalization models facilitates a comprehensive examination of the data, and provides evidence from a new, more integrated perspective, which adds to the support of ongoing discussions regarding the cognitive processes underlying the decision-making problem. These results, taken together, suggest that the hierarchical generalization modeling is a theoretically grounded and easily implementable approach to evaluate the ability of cognitive models to generalize across experimental tasks and designs.