Skip to Main Content
 

Global Search Box

 
 
 
 

Files

ETD Abstract Container

Abstract Header

Increasing the Feasibility of Multilevel Studies through Design Improvements and Analytic Advancements

Abstract Details

2019, PhD, University of Cincinnati, Education, Criminal Justice, and Human Services: Educational Studies.
Multilevel study designs are well suited for research in hierarchically structured educational settings. However, this structure, limited resources, and complex theories of teaching and learning limit the ability of educational researchers to feasibly conduct adequate studies. This three-article dissertation increases the feasibility of multilevel studies through improvements in study design and advancements in analytic approaches. The totality of this work expands the capacity of educational researchers to conduct multilevel studies. First, I extend the partial posterior predictive distribution method (p3 method) to test multilevel mediation. A variety of inferential tests are available for single and multilevel mediation but most come with notable limitations that balance tradeoffs between power and Type I error. The p3 method is a contemporary resampling-based composite approach specifically suited for complex null hypotheses. I develop the p3 method and investigate its performance within the context of two-level cluster-randomized multilevel mediation studies. The p3 method performed well relative to other mediation tests because it provides a more judicious balance of the Type I error rate and power. The method serves as a powerful alternative tool for researchers investigating multilevel mediation. Next, I investigate the robustness of statistical power under an optimal sampling framework to misspecified parameter values in cluster-randomized designs with cluster- or individual-level mediators. When planning cluster-randomized studies probing mediation, effective and efficient sample allocation is governed by several parameters. In the design stage, these parameters are typically approximated using information from prior research and these approximations are likely to deviate from the true values eventually realized in the study. The results suggest that estimates of statistical power are robust to misspecified parameter values across a variety of conditions and tests giving researchers a better understanding of the resources necessary to conduct such studies. Finally, I develop simple formulas to adjust statistical power, minimum detectable effect size, and optimal sample allocation for two-level cluster and multisite randomized designs when the outcome is subject to measurement error. Educational research studies frequently draw on fallible measures of outcomes that introduce significant measurement error. Despite the known detrimental effects of measurement error, studies are routinely planned ignoring this error. These new formulations help educational researchers account for the unfavorable effects of measurement error in the outcome when planning a study. Results suggest that ignoring outcome measurement error in the design phase can critically undermine the sufficiency and efficiency of a study and constrain the evidence studies can bring to bear on the efficacy of programs. Overall, these studies increase the feasibility of multilevel studies in educational settings and the capacity of educational research. More specifically, they improve study planning and produce more sensitive analytic approaches. The value and prevalence of multilevel studies in educational settings inherently adds importance to these advancements while the limited resources available for educational research and complexity of teaching and learning theories continue to call for methodological solutions that expand the capacity of educational researchers to conduct adequate multilevel studies.
Benjamin Kelcey, Ph.D. (Committee Chair)
Amy Farley, Ph.D. (Committee Member)
Jessaca Spybrook, Ph.D. (Committee Member)
Christopher Swoboda, Ph.D. (Committee Member)
135 p.

Recommended Citations

Citations

  • Cox, K. (2019). Increasing the Feasibility of Multilevel Studies through Design Improvements and Analytic Advancements [Doctoral dissertation, University of Cincinnati]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1560867182189418

    APA Style (7th edition)

  • Cox, Kyle. Increasing the Feasibility of Multilevel Studies through Design Improvements and Analytic Advancements. 2019. University of Cincinnati, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=ucin1560867182189418.

    MLA Style (8th edition)

  • Cox, Kyle. "Increasing the Feasibility of Multilevel Studies through Design Improvements and Analytic Advancements." Doctoral dissertation, University of Cincinnati, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1560867182189418

    Chicago Manual of Style (17th edition)