Skip to Main Content
 

Global Search Box

 
 
 

ETD Abstract Container

Abstract Header

A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process

Moody Rideout, Blaire L
http://orcid.org/0000-0001-9908-4045
http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934

Abstract Details

2017, Doctor of Philosophy (Ph.D.), Bowling Green State University, Higher Education Administration.
In 2015, the American Council on Education surveyed undergraduate admission and enrollment management leaders at 338 four-year institutions to understand holistic admissions review (Espinosa, Gaertner, and Orfield, 2015). In the report titled, Race, Class and College Access: Achieving Diversity in a Shifting Legal Landscape, 92% of selective institutions reported utilizing a holistic admissions review process and 72% cited the Gratz v. Bollinger (2003) and Grutter v. Bollinger (2003) court cases as the impetus for increasing their use (Espinosa et al., 2015). Holistic admissions review has led to admission directors relying heavily on applicant qualities beyond GPA and test scores, such as leadership and extracurricular experience (Hoover, 2013; Sandlin, 2008; Spencer, 2008). Yet, little research has been conducted on the reliability of holistic admissions as an assessment or review tool, which is inherently subjective. This study sought to examine a holistic admissions review process at one institution to simply show if variance occurred and possible explanations for it. The study analyzed reader reviews for approximately 15,000 individual undergraduate admission reviews over three years. The primary method was focused on the inter-rater reliability of the application reviews. If there was significant variance for particular reviews, there was an analysis of the content behind such reviews. Findings revealed that subjective measures of performance review, and in this case holistic admissions review, may not be susceptible to high levels of variability with 97% of reader reviews showing agreement for this study. Overall, just 3% of the subjective application reviews showed variability. The content analysis of the 208 application reviews with variance revealed 20 categories within six themes of Concern, Diversity, Extracurricular, Lacking Substance, Missing Information, and Writing Style. Additionally, female and international applicants were more represented in the variability group when comparing to applicants overall. Implications for future research and professional practice should focus on reader characteristics and experiences, quantity of readers utilized, reader review systematization, and reader hiring, training, and supervision.
Dafina-Lazarus Stewart (Advisor)
Steven Cady (Other)
Hyun Kyoung Ro (Committee Member)
Patrick Pauken (Committee Member)
161 p.
Education; Educational Leadership; Higher Education; Higher Education Administration
Admissions; Holistic Admissions; Selective Admissions; Enrollment Management; Application Readers; Admission Reviewers; Admission Raters; Inter-rater Reliability

Recommended Citations

Citations

  • Moody Rideout, B. L. (2017). A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process [Doctoral dissertation, Bowling Green State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934

    APA Style (7th edition)

  • Moody Rideout, Blaire. A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process. 2017. Bowling Green State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934.

    MLA Style (8th edition)

  • Moody Rideout, Blaire. "A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process." Doctoral dissertation, Bowling Green State University, 2017. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934

    Chicago Manual of Style (17th edition)

bgsu1491145798861934
1,128
© 2017, all rights reserved.
This open access ETD is published by Bowling Green State University and OhioLINK.