Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Adam_Combs_Dissertation_Final.pdf (10.16 MB)

ETD Abstract Container

Abstract Header

Bayesian Model Checking Methods for Dichotomous Item Response Theory and Testlet Models

Combs, Adam
http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1394808820

Abstract Details

2014, Doctor of Philosophy (Ph.D.), Bowling Green State University, Statistics.
The predominant model checking method used in Bayesian item response theory (IRT) models has been the posterior predictive (PP) method. In recent years, two new Bayesian model checking methods have been proposed that may be used as alternatives to the PP method. We refer to these as the prior-predictive posterior simulation (PPPS) method of Dey et al. (1998), and the pivotal discrepancy measure (PDM) method of Johnson (2007). These methods have shown to be effective in other Bayesian models, but have never been implemented with Bayesian IRT models. It is of practical interest to see if either of these two new methods will perform better than the PP method in assessing aspects of fit in an IRT model setting. In this dissertation, we compared the effectiveness of the PPPS and PDM model checking methods with the PP method in evaluating person fit in two-parameter normal ogive (2PN) IRT models, and overall model goodness-of-fit in 2PN testlet models. Two simulation studies were performed. The first study explored the performance of each method (PP, PPPS, and PDM) in assessing person fit, or the goodness-of-fit of an individual's set of test answers with the assumed Bayesian 2PN IRT model. Several classical person fit measures were employed under each method. We also introduced using the sum of squared Bayesian latent residuals as a person fit measure. Four different types of person miss-fit were taken from the literature, and response data sets were simulated with certain examinee's responses following these violations. We found that for most of the measures, the PPPS and PDM methods outperformed the PP method in detecting the examinee's response patterns simulated to be aberrant under the model. In particular, the sum of squared Bayesian latent residuals showed to be a very effective measure under the PPPS method. The second simulation study compares the performance of the PP method and the PPPS method in assessing the overall goodness-of-fit of a Bayesian 2PN IRT model fitted to data generated under a Bayesian 2PN testlet model with equal variance across testlets. Under the PP method we used three goodness-of-fit measures based on biserial correlations that were previously employed for checking the goodness-of-fit of a three-parameter logistic (3PL) IRT model to 3PL testlet data. For use under the PPPS method, we introduced three new goodness-of-fit measures which are calculated from posterior values of the item discrimination parameters. Data sets were simulated under four different values of testlet variance, ranging from very low to fairly high. Looking at the detection rates under the PP method, we saw that the measures performed very poorly in detecting a lack of fit of the 2PN IRT model for all data values of testlet variance. The detection rates of the new measures under the PPPS method showed to be higher than those under the PP method. However, the measures under the PPPS method only showed descent power in detecting lack of fit for large values of data generating testlet variance.
James Albert (Advisor)
Lynne Hewitt (Committee Member)
Hanfeng Chen (Committee Member)
Maria Rizzo (Committee Member)
240 p.
Statistics
Bayesian Model Checking; Posterior Predictive Model Checking; Pivotal Discrepancy Measures; Prior Predictive Posterior; Testlet Model Checking; Item Response Theory; Person Fit Checking

Recommended Citations

Citations

  • Combs, A. (2014). Bayesian Model Checking Methods for Dichotomous Item Response Theory and Testlet Models [Doctoral dissertation, Bowling Green State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1394808820

    APA Style (7th edition)

  • Combs, Adam. Bayesian Model Checking Methods for Dichotomous Item Response Theory and Testlet Models. 2014. Bowling Green State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1394808820.

    MLA Style (8th edition)

  • Combs, Adam. "Bayesian Model Checking Methods for Dichotomous Item Response Theory and Testlet Models." Doctoral dissertation, Bowling Green State University, 2014. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1394808820

    Chicago Manual of Style (17th edition)

bgsu1394808820
1,024
© 2014, all rights reserved.
This open access ETD is published by Bowling Green State University and OhioLINK.