The formative assessment literature has unanimously heralded the benefits of the diagnostic use of assessment to inform curriculum and instruction and improve student performance and achievement. Research in this area has primarily focused on traditional formative assessment practices. More recently, research is beginning to examine the effectiveness of technology-based formative assessment, with the latest studies of this mode formative assessment beginning to replicate the traditional findings. The current study examined one computerized/online formative assessment program, the Diagnostic Online Reading Assessment (DORA), and its relationship to a summative state proficiency test, in addition to examining the multilevel influence of teacher use of this technology-based mode of formative assessment on student DORA growth.
Existing state test data from one school district in Colorado and existing DORA scores were obtained. In addition, teacher survey data from the same Colorado school district and across the United States were collected online. Student data from grades 3 through 11 and teacher survey data were analyzed via Hierarchical Linear Growth Modeling and Rasch Analysis to investigate the following three main objectives: (1) Examining if DORA growth is related to state test score growth, (2) Developing a behavioral frequency measure of teacher use of computerized/online formative assessment, the Online Formative Assessment Survey (OFAS), and (3) Investigating the relationship between the OFAS and student DORA growth.
Specific to the first objective, it was found that DORA subtest growth was significantly and positively related to state reading test score growth. For the second objective, it was found that a psychometrically sound measure of teacher computerized/online formative assessment practices can be developed. The results rendered a 50-question measure focusing on all elements of teacher use of computerized/online formative assessment, and 10-question measure concentrating solely on how teachers use the results from the computerized/online formative assessment program. In the third objective, it was found that both the 50-question and 10-question OFAS were not significant, positive predictors of student DORA score growth.
Although the validation of OFAS scores was not supported, future research should continue to define this theoretical network of relationships, and maintain the measure revision and validation process. The Rasch results provide psychometric support for this newly developed measure, specifically the focus on using the online formative assessment results. Internet-mediated assessment is becoming commonplace in the classroom, and is more frequently being used to replace traditional modes of assessment. The need to examine the extent to which these methods are educationally sound is in high demand. Results from this study can support administrative demands for more efficient, technology-based ways to encourage teachers to use this mode of formative assessment, and in turn, meet state standards and increase student achievement.