Skip to Main Content
 

Global Search Box

 
 
 
 

Files

ETD Abstract Container

Abstract Header

Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension

Workman, Jeffrey Dale

Abstract Details

2010, Doctor of Philosophy, Ohio State University, Agricultural and Extension Education.
Documenting evidence of impact demonstrates that organizations and programs are making a difference and/or accomplishing something positive. The results of a program must ultimately change people’s attitudes or behavior, or benefit society in other ways (Diem, 1997). As competition increases and the availability of financial resources decreases, the U.S. Cooperative Extension System must document evidence of impact to demonstrate that it’s worth funding. In general, Extension programs which utilize program evaluation in the form of outcome studies do not document impact by collecting real evidence of behavior change and/or greater end results that benefit societal conditions through social, economic, civic, or environmental improvement. Traditionally, the criticism of evaluation in Extension has been that evaluators have simply reported on the number and demographics of the people who participated (Gentry-Van Laanen & Nies, 1995). The purpose of this study was to explore and describe the level of evidence of impact collected through program evaluation (outcome studies) by the Cooperative Extension System as published in the Journal of Extension (JOE); and to examine any potential relationships between the documentation of evidence of impact with reach (size of audience), richness (detail and depth of interaction), and geographic scope. Articles reviewed in JOE were those listed under the headings of ‘Feature Articles’ and ‘Research in Brief’ during 1965-69, 1975-79, 1985-89, 1995-99, and 2005-09. This study utilized a form of quantitative content analysis where data was collected and coded using qualitative methods and analyzed via quantitative techniques. Two evaluation tools were used for this study. Bennett’s Hierarchy was utilized because it was considered to be the simplest, easiest to understand, and contained the most applicable description of different levels of evidence of impact. The University of Wisconsin-Extension logic model format was also included because it was considered to be the more contemporary and predominant tool currently used when documenting and describing results from Extension programming. There were a total of 1,246 articles investigated in which 302 were selected for data collection. This investigation clearly showed that the number of evaluative studies with outcome data has increased going from 16 articles in the 1960s to 150 articles in the 2000s. The findings indicated that 88.5% of the articles documented evidence above the level of participation, almost two-thirds were measuring outcomes, about one-third were measuring evidence of substantial impact, and 5.6% documented “real or actual” impact. The relationships of reach of richness with level of evidence were low at best, and the relationship with geographic scope was very low. It is highly likely that Extension programs make a tremendous positive difference in the lives of individuals and their communities and Extension greatly benefits society as a whole, but more program evaluation with evidence of higher-level impact is needed to support this argument. It is also likely that funding becomes more limited and much more competitive in the future. Those organizations who have real documented evidence of their worth and merit will be the ones that receive the funding and continue to exist in the future.
Scott Scheer, PhD (Advisor)
Joe Gliem, PhD (Committee Member)
Karen Bruns, PhD (Committee Member)
174 p.

Recommended Citations

Citations

  • Workman, J. D. (2010). Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544

    APA Style (7th edition)

  • Workman, Jeffrey. Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension. 2010. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544.

    MLA Style (8th edition)

  • Workman, Jeffrey. "Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension." Doctoral dissertation, Ohio State University, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544

    Chicago Manual of Style (17th edition)