Skip to Main Content
Frequently Asked Questions
Submit an ETD
Global Search Box
Need Help?
Keyword Search
Participating Institutions
Advanced Search
School Logo
Files
File List
osu1291120544.pdf (1.63 MB)
ETD Abstract Container
Abstract Header
Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension
Author Info
Workman, Jeffrey Dale
Permalink:
http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544
Abstract Details
Year and Degree
2010, Doctor of Philosophy, Ohio State University, Agricultural and Extension Education.
Abstract
Documenting evidence of impact demonstrates that organizations and programs are making a difference and/or accomplishing something positive. The results of a program must ultimately change people’s attitudes or behavior, or benefit society in other ways (Diem, 1997). As competition increases and the availability of financial resources decreases, the U.S. Cooperative Extension System must document evidence of impact to demonstrate that it’s worth funding. In general, Extension programs which utilize program evaluation in the form of outcome studies do not document impact by collecting real evidence of behavior change and/or greater end results that benefit societal conditions through social, economic, civic, or environmental improvement. Traditionally, the criticism of evaluation in Extension has been that evaluators have simply reported on the number and demographics of the people who participated (Gentry-Van Laanen & Nies, 1995). The purpose of this study was to explore and describe the level of evidence of impact collected through program evaluation (outcome studies) by the Cooperative Extension System as published in the Journal of Extension (JOE); and to examine any potential relationships between the documentation of evidence of impact with reach (size of audience), richness (detail and depth of interaction), and geographic scope. Articles reviewed in JOE were those listed under the headings of ‘Feature Articles’ and ‘Research in Brief’ during 1965-69, 1975-79, 1985-89, 1995-99, and 2005-09. This study utilized a form of quantitative content analysis where data was collected and coded using qualitative methods and analyzed via quantitative techniques. Two evaluation tools were used for this study. Bennett’s Hierarchy was utilized because it was considered to be the simplest, easiest to understand, and contained the most applicable description of different levels of evidence of impact. The University of Wisconsin-Extension logic model format was also included because it was considered to be the more contemporary and predominant tool currently used when documenting and describing results from Extension programming. There were a total of 1,246 articles investigated in which 302 were selected for data collection. This investigation clearly showed that the number of evaluative studies with outcome data has increased going from 16 articles in the 1960s to 150 articles in the 2000s. The findings indicated that 88.5% of the articles documented evidence above the level of participation, almost two-thirds were measuring outcomes, about one-third were measuring evidence of substantial impact, and 5.6% documented “real or actual” impact. The relationships of reach of richness with level of evidence were low at best, and the relationship with geographic scope was very low. It is highly likely that Extension programs make a tremendous positive difference in the lives of individuals and their communities and Extension greatly benefits society as a whole, but more program evaluation with evidence of higher-level impact is needed to support this argument. It is also likely that funding becomes more limited and much more competitive in the future. Those organizations who have real documented evidence of their worth and merit will be the ones that receive the funding and continue to exist in the future.
Committee
Scott Scheer, PhD (Advisor)
Joe Gliem, PhD (Committee Member)
Karen Bruns, PhD (Committee Member)
Pages
174 p.
Subject Headings
Adult Education
;
Agricultural Education
;
Agriculture
;
Educational Evaluation
Keywords
Program Evaluation
;
Impact
;
Evidence
;
Extension
;
Bennett's Hierarchy
;
Logic Model
Recommended Citations
Refworks
EndNote
RIS
Mendeley
Citations
Workman, J. D. (2010).
Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension
[Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544
APA Style (7th edition)
Workman, Jeffrey.
Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension.
2010. Ohio State University, Doctoral dissertation.
OhioLINK Electronic Theses and Dissertations Center
, http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544.
MLA Style (8th edition)
Workman, Jeffrey. "Exploring the Level of Evidence of Impact Collected through Program Evaluation by Cooperative Extension as Published in the Journal of Extension." Doctoral dissertation, Ohio State University, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=osu1291120544
Chicago Manual of Style (17th edition)
Abstract Footer
Document number:
osu1291120544
Download Count:
1,232
Copyright Info
© 2010, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.