Skip to Main Content
Frequently Asked Questions
Submit an ETD
Global Search Box
Need Help?
Keyword Search
Participating Institutions
Advanced Search
School Logo
Files
File List
B. Moody Rideout FINAL Dissertation Manuscript.pdf (640.66 KB)
ETD Abstract Container
Abstract Header
A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process
Author Info
Moody Rideout, Blaire L
ORCID® Identifier
http://orcid.org/0000-0001-9908-4045
Permalink:
http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934
Abstract Details
Year and Degree
2017, Doctor of Philosophy (Ph.D.), Bowling Green State University, Higher Education Administration.
Abstract
In 2015, the American Council on Education surveyed undergraduate admission and enrollment management leaders at 338 four-year institutions to understand holistic admissions review (Espinosa, Gaertner, and Orfield, 2015). In the report titled, Race, Class and College Access: Achieving Diversity in a Shifting Legal Landscape, 92% of selective institutions reported utilizing a holistic admissions review process and 72% cited the Gratz v. Bollinger (2003) and Grutter v. Bollinger (2003) court cases as the impetus for increasing their use (Espinosa et al., 2015). Holistic admissions review has led to admission directors relying heavily on applicant qualities beyond GPA and test scores, such as leadership and extracurricular experience (Hoover, 2013; Sandlin, 2008; Spencer, 2008). Yet, little research has been conducted on the reliability of holistic admissions as an assessment or review tool, which is inherently subjective. This study sought to examine a holistic admissions review process at one institution to simply show if variance occurred and possible explanations for it. The study analyzed reader reviews for approximately 15,000 individual undergraduate admission reviews over three years. The primary method was focused on the inter-rater reliability of the application reviews. If there was significant variance for particular reviews, there was an analysis of the content behind such reviews. Findings revealed that subjective measures of performance review, and in this case holistic admissions review, may not be susceptible to high levels of variability with 97% of reader reviews showing agreement for this study. Overall, just 3% of the subjective application reviews showed variability. The content analysis of the 208 application reviews with variance revealed 20 categories within six themes of Concern, Diversity, Extracurricular, Lacking Substance, Missing Information, and Writing Style. Additionally, female and international applicants were more represented in the variability group when comparing to applicants overall. Implications for future research and professional practice should focus on reader characteristics and experiences, quantity of readers utilized, reader review systematization, and reader hiring, training, and supervision.
Committee
Dafina-Lazarus Stewart (Advisor)
Steven Cady (Other)
Hyun Kyoung Ro (Committee Member)
Patrick Pauken (Committee Member)
Pages
161 p.
Subject Headings
Education
;
Educational Leadership
;
Higher Education
;
Higher Education Administration
Keywords
Admissions
;
Holistic Admissions
;
Selective Admissions
;
Enrollment Management
;
Application Readers
;
Admission Reviewers
;
Admission Raters
;
Inter-rater Reliability
Recommended Citations
Refworks
EndNote
RIS
Mendeley
Citations
Moody Rideout, B. L. (2017).
A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process
[Doctoral dissertation, Bowling Green State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934
APA Style (7th edition)
Moody Rideout, Blaire.
A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process.
2017. Bowling Green State University, Doctoral dissertation.
OhioLINK Electronic Theses and Dissertations Center
, http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934.
MLA Style (8th edition)
Moody Rideout, Blaire. "A Study of the Inter-rater Reliability of University Application Readers in a Holistic Admissions Review Process." Doctoral dissertation, Bowling Green State University, 2017. http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1491145798861934
Chicago Manual of Style (17th edition)
Abstract Footer
Document number:
bgsu1491145798861934
Download Count:
1,128
Copyright Info
© 2017, all rights reserved.
This open access ETD is published by Bowling Green State University and OhioLINK.