what you should expect from your evaluator: promising practical practices
DESCRIPTION
What You Should Expect from your evaluator: promising practical practices. July 28, 2011 Hi-TEC Conference, San Francisco. Three perspectives. Experienced ATE Program manager: Lara Smith Experienced community college professional and researcher: Dr. Jerry Somerville - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/1.jpg)
WHAT YOU SHOULD EXPECT FROM YOUR
EVALUATOR: PROMISING PRACTICAL PRACTICESJuly 28, 2011 Hi-TEC Conference, San Francisco
![Page 2: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/2.jpg)
Three perspectives Experienced ATE Program manager: Lara
Smith Experienced community college
professional and researcher: Dr. Jerry Somerville
Experienced ATE evaluator and researcher: Dr. Norena Norton Badway
![Page 3: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/3.jpg)
Who is in attendance? Community College University High School Workforce Development Advocate Trade Organization Industry Professional Technician ATE Personnel ATE Project or Center Director Program Evaluator Researcher
![Page 4: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/4.jpg)
What is Evaluation? It is a form of research
The results are to be used
Its purpose is to educate Assist in making informed decisions
![Page 5: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/5.jpg)
Information Needs of ATE Programs
Using a Delphi Process (an elaborate survey) we asked ATE leaders about the most important information needs.
Purpose: to identify research topics that researchers could study that would produce meaningful results for practitioners.
We hoped to overcome the issue that results from research/ evaluation are not used in practice.
![Page 6: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/6.jpg)
The first thing we learned…
![Page 7: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/7.jpg)
A Few of Our Findings Program directors desire meaningful and
usable results for program improvement Practitioners feel that from their experience
they know what works and what doesn't They don’t have the time or expertise to
conduct research They want someone else to collect, analyze
and synthesize the data and then disseminate the results in a meaningful form
![Page 8: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/8.jpg)
Our Belief Practitioners are the primary users of the
information generated from evaluation/research.
If it is to be useful, practitioners need to
be actively involved in working with evaluators/researchers.
![Page 9: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/9.jpg)
Dissemination:Suggestions by ATE Leaders Offer sessions at ATE/HI-TEC types of
conferences Make more use of Webinars Publish a periodical that digests both
promising practices and unsuccessful methods and presents them in briefs.
Create a listserv for ATE project and center directors so they can communicate with each other.
![Page 10: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/10.jpg)
Dissemination Beyond ATE: Community College Related
Community College Journal of Research and PracticeServes researchers and practitioners in higher education and the behavioral and social sciences. Journal for Applied Research in Community Colleges Serves institutional research and planning professionals in 2-year, postsecondary educational institutionsThe Community College Review Deals with all aspects of community college administration, education, and policy.
![Page 11: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/11.jpg)
Dissemination Beyond ATE: STEM Related Journals
Journal of STEM EducationEmphasizes real-world case studies that focus on issues that are relevant and important to STEM practitionersResearch in Science & Technology EducationPublishes original research from throughout the world dealing with science education and/or technological education
![Page 12: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/12.jpg)
Dissemination Beyond ATE: CTE Related Journals
Journal of Career and Technical EducationServes as a forum for discussion of philosophy, theory, practice, and issues in career and technical education.
Career and Technical Education ResearchPublishes articles that examine research in vocational/career and technical education, career development, human resource development, career issues in grades K-12, postsecondary education, adult and lifelong learning, and workforce education.
![Page 13: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/13.jpg)
WHY do Evaluation? ATE requires it Acts as a “mirror” so you can see what
you are doing Lends legitimacy/ evidence to your
claims of achievement or challenges Is a tool for professional development for
staff and stakeholders [measuring impact on student learning]
Offers collaboration for dissemination
![Page 14: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/14.jpg)
Who Should You Select as Evaluator?
Match personally and professionally Experienced with ATE and CCs Applies evidence-based/ research theoretic
framework Moves beyond “body counts” Analyzes, rather than audits Partners, rather than judges Provides sample report(s) Evalua|t|e website
![Page 15: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/15.jpg)
WHEN Should You Engage Evaluator?
![Page 16: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/16.jpg)
Measuring IMPACT Impact on student learning Impact on faculty learning
Impact on faculty technical knowledge Impact on pedagogy Impact on curriculum design
Impact on organizational learning Structure of program/ schedule of courses Expanding access and success
![Page 17: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/17.jpg)
Realities for an ATE Program Establishing the evaluator relationship Time Budget Time
![Page 18: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/18.jpg)
Establish Scope of Work (SOW) Accompanies contract Defines activities;
deliverables Allows evaluator to invoice Allows project to pay
![Page 19: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/19.jpg)
Cost Proposal Start with Scope of Work Decide on hourly or fixed payment Establish number of site visits and/or
other travel Establish number and type of reports
![Page 20: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/20.jpg)
Budget ExampleEvaluation expenses$50,000 fixed fee + $9,965 expenses$59,965
Where are the negotiation points?
6.7%7
Project Budget$900,000 ÷ 3$300,000 per year
![Page 21: What You Should Expect from your evaluator: promising practical practices](https://reader035.vdocuments.site/reader035/viewer/2022062410/56816240550346895dd27856/html5/thumbnails/21.jpg)
For more information Norena Norton Badway
[email protected] Jerry Somerville
[email protected] Lara Smith