Evaluating for Impact Learning Circle Project

Faculty Presentation

Name: Ellen Rowe Title: Community & Leadership Development

Ideal Distribution of Evaluation Capacity in Public Service Organizations

(Douglah, Boyd, & Gundermann, 2003)

Content Modules

• Program Planning • Focusing an Evaluation• Evaluation Questions and Designs• Evaluation Methods• Collecting and Handling Data• Analyzing and Interpreting Data• Communicating Evaluation Results

How to Find the Educational Content

http://www.national4-hheadquarters.gov/comm/4h_ydresearch.htm (Educational Content and Self-Assessment Tool)

Goals of Evaluating for Impact Project1.Provide learning and practice opportunities to develop educators to the practitioner level2.Provide coordinated education in program evaluation that is consistent in scope and depth across the country3.Provide opportunities for learning and practice through the use of evaluation learning circles and collaborative learning environments- both on-line and on-site4.Provide customized content that is useful for 4-H educators, thus facilitating the application of what is learned

Input: Facilitated Project in Vermont 1) Recruit eight to ten 4-HYD professionals to participate in a learning community focused on the modules 2) Describe their plan for engaging the learning community through electronic and/or in-person communication 3) Describe how they implemented the learning community 4) Describe how users perceived different parts of the modules and their understanding of concepts (I would recommend 3 or 4 main topics/concepts/etc. as selected by your Evaluating for Impact team) 5) Report back a 2-3 page brief (due May 1, 2009) and participate in a follow-up teleconference after the learning communities completed their work with the modules

Vermont Evaluation Mentoring Project

• Complete a self-assessment• Review results:

- for module areas to strengthen (ability/ capacity)- for module areas important to your job - for module areas important to the organization

• Select modules to focus learning upon

Overview: I would like to receive input from A) Parents of Vermont 4-H members as to what SET skills they feel their children need and B) Parents of older Vermont 4-H members whether or not Vermont 4-H prepared their children for a career in SET. Goal to be Achieved: To create a 4-H SET Parent Survey to be administered at 2009 Vermont 4-H State Day to use as a base line for future evaluation of VT 4-H SET programming. Module 2 Actions – “Focusing on Evaluation” Date Outcome Action Resources 4/13/09 Understand process & outcome

evaluation Course study/readings SAMMIE 2

4/13/09 Develop evaluation Qs Course study/readings SAMMIE 5 4/13/09 Identify indicators of change Course study/readings UWEX Logic Model

Tutorial – module 1, section 7

Module 3 Actions – “Evaluation Questions & Design” Date Outcome Action Resources 4/20/09 Understand types of evaluation Qs;

Understand how to refine & test Qs Course study/readings SAMMIE 5

Results & Reflections• Up to 12 hours per week for mentor• Comfortable with facilitated learning from YDPA

participation• Independent Study method appreciated with staff

busy schedules• Members were actively engaged; conference calls

regularly at 100% participation• Six of the Learning Action Plan addressed module

3 (Evaluation Questions & Designs); sense of collective inquiry, shared freely

Goal: To create better questions for the evaluation I conduct; ultimately to get richer data that I can use to support program outcomes.

“Responses to 2010 Teen Congress revealed that many kids are challenged by having to speak to people they don’t know and Teen Congress helps them to get over this.

“Conducted an evaluation on the experience of Teen Board members. I used the findings from this evaluation to help launch 4-H Leadership Clubs in afterschool programs.”

Goal: To create a 4-H SET parent Survey to be administered at 2009 VT 4-H State Day to use as a baseline for future evaluations of VT 4-H SET programming. “I created a survey and implemented it at 2009’s State Day. As a result of the survey, 1)VT 4-H created the 4-H SET web page to inform & promote program offerings, trainings and 4-H Science resources available to 4-H volunteers, teachers and home school families; and 2) a 4-H Science “what’s new” type of article has appeared in every VT Clovergram since June 2009.

“Plan to replicate the survey at State Day, May 2011.”

Goal: To create an evaluation for the State Hippology Contest to better assess learning related to biosecurity, life skills, and SET abilities.

“Incorporated a pre-test to bring focus of participants to these important topics: do you like science? what are life skills?; what is biosecurity?

“Post event assessment addressed: what did you learn that was science-like?; what biosecurity steps will you use at home?; and what life skills did you use today?”

“Changing the evaluation in 2011 to make it a little more youth friendly.”

Goal: Create an evaluation for the Horse Clinic that will indicate the level of learning taking place in the 2 day Clinic.

“Survey data was shared with the 4-H Horse Leaders and the new coordinator for the 4-H Horse Clinic.

“Plans are being made for changes in the 2011 Clinic that will reflect some of the concerns identified in the 2009 survey. The Clinic has been primarily for riding lessons in the past. This year’s Clinic will include activities for 4-H members in the Horseless Horse project. There will also be more structured time for group activities designed to foster teamwork, communication and acceptance of differences.”

Baughman, S., Arnold, M., Boyd, H.H., Franz, N.K., Mead, J., Rowe, E., & Silliman, B. (2010). Evaluating for Impact professional development curriculum delivery through learning circles. Journal of Extension. [On-line Serial], 48 (3). http://www.joe.org/joe/2010june/tt3.php