developing evaluation evidence revised3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... ·...

16
The National Center for the Evaluation of Educational Leadership Preparation and Practice Developing Evaluation Evidence: A Formative and Summative Evaluation Planner for Educational Leadership Preparation Programs

Upload: others

Post on 02-Sep-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

TheNationalCenterfor

theEvaluationof

EducationalLeadership

PreparationandPractice

DevelopingEvaluationEvidence:

AFormativeandSummativeEvaluationPlannerforEducationalLeadershipPreparation

Programs

Page 2: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

ThispublicationwasdevelopedandproducedbytheNationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice(NCEELPP)(www.edleaderprep.org).Thepurposeofthecenteristomakeavailablevalidandreliableevaluationresearchtools,methodsandtrainingmaterialsandstrategiesforleadershippreparationprogramsaswellasasystematicprocessforcollectingandanalyzingstatedataondegreesandcertificationbyinstitution,andcareeradvancementandschoolprogressbygraduatesandinstitutions.Thecenterprovidestools,training,technicalassistanceandsupportforleadershippreparationprograms.

CenterDirectors:

MargaretTerryOrr

AndreaK.Rorrer

MichelleD.Young

TechnicalandSeniorResearchAssociate

CoriA.Groth

ThisreportwaspreparedbyMargaretT.Orr,MichelleD.YoungandAndreaK.Rorrer.Thisdocument,DevelopingEvaluationEvidence,aswellasotherresourcematerialsforleadershipevaluationareavailablefreeofchargeforreviewordownloadfromthecenter’swebsite:www.edleaderprep.org.

TheCenterissupportedbytheUniversityCouncilforEducationalAdministration(UCEA)andtheUtahEducationalPolicyCenter.

Page 3: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

TableofContentsFormativeandSummativeEvaluationPlanningforLeadershipPreparationPrograms..............................2

EvaluationPlanningGuide.......................................................................................................................3

Pre-Conditions......................................................................................................................................4

ProgramQualityFeatures....................................................................................................................4

FormativeAssessmentsofCandidateLearning....................................................................................4

SummativeAssessmentsofCandidateLearning..................................................................................5

CareerOutcomes.................................................................................................................................5

LeaderPractices...................................................................................................................................6

StaffandSchoolPractices....................................................................................................................6

StaffandSchoolEffects........................................................................................................................6

StudentOutcomes................................................................................................................................7

EvaluationPlanningGuideAlignmenttoNationalAccreditationEvaluationExpectations.........................7

EvaluationPlanningWorksheet...................................................................................................................9

References..................................................................................................................................................13

Figure1.EvaluationPathwayforPreparationProgramsPreparationPrograms.........................................3

Table1.EvaluationPlanningGuideWorksheet...........................................................................................9Table2.EvaluationPlanningGuideBlankWorksheetforProgramUse....................................................12

Page 4: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page2

FormativeandSummativeEvaluationPlanningforLeadershipPreparationProgramsEducationalleadershippreparationprogramsarecurrentlyengagedinformativeandsummativeevaluationoftheircandidatesandgraduatesforavarietyofpurposesanduses.Theseincludeto:

• Providerelevantandtimelyinformationongraduatesandalumnioutcomestoascertainprogrameffectiveness

• Comparegroupsofgraduates’experiencestodeterminebenefitsofprogramdifferences

• Compareprogramfeaturesanddeliverytypewithotherprogramsregionallyandnationallyforbenchmarking

• Identifyareasforprogramandcourseimprovement• Makethecaseforprogramresourcesandsupport• Researchtherelationshipbetweenprogramdesign

anddelivery,graduateoutcomes,andschoolimprovementwork

Tofacilitateplanninganddatacollection,thisformativeandsummativeevaluationplannerhasbeencreated.Theplannerisorganizedaccordingtohowprograminputsandoutcomeshavebeenconceptualizedandvalidatedintheevaluationresearchonleadershippreparationprograms.Thisprogramevaluationhasmultipleuses,includingthosecitedinthetextbox.

ThePlannerincludes:

• Aconceptualmodelofthelinkbetweenleadershippreparationandoutcomes.

• Aguideforidentifyingevaluationevidence.• Anevaluationplanningworksheet.

Eachprogramisguidedbyitsowntheoryofactionorprogramtheory,whichconnectsitschoicesinprogramcontent,delivery,anddesigntoexpectedoutcomes.Weiss(1988)definesprogramtheoryas“thesetofbeliefsthatunderlieaction”(p.55)andexplainsthattheserepresentthemechanismsthatmediatebetweendeliveryofaprogramandtheintendedoutcomes(Weiss,1998).Inplanningforevaluation,programofficials,therefore,willneedtodetermineboththeoutcomestheyexpectaswellastheattributesoftheprogramtheythinkaremostinfluential.Thisplannershouldhelpprogramofficialsmaketheseselectiondecisionsbyidentifyingwhattheywanttomeasureandthesourcesofevidencetheyplantouse.Aswell,

ProgramEvaluation

UsesIdentifyFormativeandSummativeAssessments

IdentifyMeasuresandOutcomes(e.g.,programandparticipantoutcomes)

EvaluatetheRelationshipbetweentheProgram

AttributeandtheOutcome

UseDataforPreparationProgramImprovement

Page 5: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

therecommendedevaluationevidencecategoriesmayhelpprogramofficialsconsiderevaluationoptionstheyhadnotanticipated.

Figure1belowillustratesanEvaluationPathwayforPreparationPrograms.ThePathwaywasconstructedusingavailableevaluationresearchvalidatedthroughseveralstudies.AsindicatedinthePathway,therearetwocategoriesofprograminput,threecategoriesofinitialgraduateoutcomes,andtwocategoriesofexpectedschooloutcomes.Thismodelservesasablueprintforevaluationplanning,asexplainedbelow.

Figure1.EvaluationPathwayforPreparationProgramsPreparationPrograms

Programsneedawaytomeasureandtrackeachprogramattributeandoutcomeofinterest.Thisplannerenablesprogramstoidentifywhattheywanttomeasureandhowthesesourcesofevidencerelatetotheirprogramastheyselecttheirformativeandsummativeevaluationassessments.

EvaluationPlanningGuideTheEvaluationplanningguideincludesasetofrecommendedsourcesofevaluationevidenceforeachkindofmeasure,asindicatedinFigure1.Itincludestypesofdatatobecollectedandrecommendedoravailablesourcesofevidence,includingtheInitiativeforSystematicProgramImprovementthroughResearchinEducationalLeadership(INSPIRE)SuiteofSurveyinstruments,whichwasspecificallydesignedforthesepurposes.(Seehttp://www.edleaderprep.org/).Theplanningguidealsoincludessuggestedtimingfordatacollection.

Preconditions:ProgramParticipantsPriorExperiences

ProgramQualityFeatures:LeadershipProgramExperiences

FormativeandSummativeLearningOutcomes

CareerOutcomes

LeaderPractices

LeadershipImpactonStaff,SchoolPractices,andSchoolCommunity

LeadershipImpactonSchoolandStudentPerformance

Page 6: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page4

Thecategoriesandtheirdefinitionsareasfollows:

Pre-Conditions(e.g.INSPIREPreparationProgramSurvey&GraduateSurvey).Forthepurposesofevaluatingleadershippreparation,thepre-conditionsarethecharacteristicsandqualitiesthatcandidateshavepriortotheirprogramexperience.Someprogramsestablishselectioncriteriathatsetlimitsonthesepre-conditions(suchasthenumberofyearsofpriorteaching,theinstructionaleffectiveness,andpriorleadershipexperiences).Someprogramsstrivetorecruitcandidatesthathelptodiversifythefieldorcreatemoreequitableaccesstoleadershippreparation,basedongender,race,orethnicity.Afinaltypicalpre-conditionhastodowiththecandidates’affiliationwithalocaldistrictandthatdistrict’srelationshipwiththeleadershippreparationprogram(intheformofreferral,collaborationandfinancialsupport).Thus,primarypre-conditionsinclude:

• demographiccharacteristicsofthecandidate.• Educational,professional,andleadershipexperiencesandaccomplishments.• districtsupport.

ProgramQualityFeatures(e.g.INSPIREPreparationProgramSurvey&GraduateSurvey).Priorresearchhasunderscoredthequalityfeaturesofleadershippreparationprogramsandtheirinfluenceongraduateoutcomes(DarlingHammond,Meyerson,LaPointe,&Orr,2009;Jackson&Kelley,2002).Theseinturnrepresentprogramdimensionsandcanbeusedtomeasurehowtheyvarytheircontent,deliveryandquality.Thesedimensionsinclude:

• philosophyortheoryofaction• curriculumandcoursecontent(relationshiptonationalstandards,focuson

instructionalleadershipandotherprogrampriorities)• instructionalapproaches(suchasuseofproblem-basedlearning,casestudies

andactionresearch;technologysupportedlearning)• internship(length,focusandquality)• candidatesupportanddevelopment(includingcohortstructuresandadvisement)• assessments(includingexams,portfolioassessments,culminatingprojects,

internshipperformanceassessments,andstatecertificationassessments)• post-programsupport(suchasseminars,mentoringandcoaching,andjob

placementassistance).• faculty(whetherdedicatedtotheprogram,tenured,research-engaged,and

havingschoolanddistrictleadershipexperience)• organizationalsupports(logistics,accessibility,andeaseofuse).

FormativeAssessmentsofCandidateLearning(e.g.INSPIREPreparationProgramSurvey).Ascandidatesprogressthroughtheircourseofstudy,programfacultyandofficials

Page 7: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

needtoassesslearningandskilldevelopmentforfeedbackandcontinuousimprovement.Formativeandinterimassessmentsprovideinformationthatcanbeusedtoidentifycandidates’skillsandcompetenciesthatmayneedfurtherdevelopmentaswellaswaysinwhichtosupportsuchdevelopmentpriortocompletionoftheprogram.Thesecanincludecourse-relatedorinternship-relatedassignments.Itisalsoimportanttodeterminehowtheseassignmentsareevaluated(suchaswitharubricorotherratingtool),andhowresultsaretracked.

• course-relatedknowledgeassessments• skillspecificassessments• dispositionalassessments• internship-relatedassessments• standards-basedtasksandprojects• mid-programassessments.

SummativeAssessmentsofCandidateLearning(e.g.INSPIREPreparationProgramSurvey&GraduateSurvey).Atthecompletionofaprogram,candidates’learningandskilldevelopmentcanbeassessedthroughavarietyofculminatingassignmentsandproducts,whicharethenevaluatedusingarubricorotherratingtool,andthroughstandardizedleadershipassessmentssuchasstateandnationalexams.Summativeassessmentsaredesignedtoassesscandidates’attainmentofprogramdefinedcompetenciesandreadinessforlicensureorcertificationandforinitialleadershippositions.Theresultscanbecompiledaspartoftrackingindividualcandidatesandsummarizedbygroupandprogram.

CareerOutcomes(e.g.INSPIREGraduateSurveyandLeaderinPracticeSurvey).Aprimaryintentionofeducationalleadershippreparationprogramsistoinfluencethecareeradvancementofcandidates,includingthenature,timingandefficacyofsuchadvancement.Dependinguponaprogram’smissionandpurpose,thefocusshouldbeontheextenttowhichcandidatesbecomeschoolanddistrictleadersandareabletoadvancefairlyrapidlyintosuchpositions.Programsneedtotrackgraduates’careersovertimeforthetypesofleadershippositionsassumed,thelengthoftimeinandbetweenpositions,andtheeaseorchallengeofgainingadvancement.Also,programassessmentneedstoincludetheextenttowhichprogramsareyieldingequitableoutcomesbasedongender,race/ethnicityandotherrelevantdemographiccharacteristics.Thiscanbethroughself-reportedinformationorasobtainedfromdistrictorstateemploymentinformation,andshouldincludebothschoolanddistrict,andsupervisoryandnonsupervisoryleadershippositions.Suchmeasuresinclude:

• typeofposition• lengthoftimetoadvancetoeachleadershipposition• retentioninleadershipposition

Page 8: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page6

• easeofadvancementintoleadershippositions.

LeaderPractices(e.g.INSPIRELeaderinPractice&360survey).Bydefinition,leadershippreparationprogramsaredesignedtodeveloptheskillsandcapacitiesofeducationalleaders,whichbecomemostevidentintheirpracticesasschoolanddistrictleaders.Whilesomeleadershipskillsandcapacitiesaredevelopedbyallprograms(particularlythosedefinedbynationalleadershipstandards),someprogramsemphasizecertainskillsoverothersanddevelopadditionalskillsthroughtheirprogramcontentanddelivery.Variousprincipalassessmenttoolsandsurveysaredesignedtodocumentprincipals’useofeffectiveleadershippractices,asreportedbythemselvesorothers,andcanbeusedaspartofaprogramevaluationsystem.Theseinclude:

• principalpracticesurveys(e.g.INSPIRE–LeaderinPracticeSurvey)• supervisorsatisfactionsurveys• 360-degreefeedbackassessmentsonleadershippracticesbysupervisors,

teachersandothers(e.g.INSPIRE360Survey)• Principalevaluationsystems,suchasVal-Ed

(http://www.discoveryeducation.com/products/assessment/val_ed.cfm)(Goldring,Porter,Murphy,Elliot,&Cravens,2007)

Theiruse,however,shouldreflectalignmentwiththepreparationprogram’sprioritiesanddesign,andbeusedtoillustratestrengthsandgapsinprograms.

StaffandSchoolPractices(e.g.INSPIRE360survey–LeadershipPractices&SchoolConditionssections).Asresearchoneffectiveleadersshows,principalshavetheirgreatesteffectonstudentlearningthroughtheirworkwithteachersandinorganizingschoolconditionstooptimizingteachingandlearning(Leithwood&Jantzi,2008;Robinson,Lloyd,&Rowe,2008).Therefore,evaluatingtheeffectsofpreparationonleadershippracticesshouldlookfirstatchangesintheseareas.Suchchangeswouldincludechangesinteachers’instructionalpractices,supportofstrugglingstudents,andcollaborativeworkwitheachothertoimprovestudentlearning.Otherchangeswouldbemoreorganizationalintermsofimprovingtheavailabilityofrelevantinstructionalresources,betteruseofschooltimeandfacilities,staffprofessionaldevelopment,improveduseofstudentdatatoguideimprovement,andcoherenceofprogramsandservicesinsupportingstudentlearning.Assessingtheseimprovementscanbedonethroughprincipal,teacherandsupervisorsurveys.

StaffandSchoolEffects(e.g.INSPIRE360survey–SchoolConditionssection).Asschoolconditionsimproveandstaffmembersmakeprogressonimprovingteachingandlearning,thereshouldbeculturalandclimateeffectsonstudents,staffandthelargerschoolcommunity.Theseeffectsinclude:

Page 9: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

• studentattendanceandpositivebehavior• studentengagementandacademiceffort• teacherattendanceandrespectfultreatmentofstudentsandcolleagues• teacherengagementandacademicchallenge• improvedschoolclimate• distributedandcollaborativeleadership• improvedparentparticipation.

Student,teacher,principalandsupervisorsurveys,interviews,andotherfeedbackmechanismsareamongthebestmethodsfordeterminingtheseeffects.

StudentOutcomes.Theultimateimpactofleadershippreparationonleadershipeffectivenessisdeterminedbythedegreetowhichstudentachievementimproves.Suchimprovements,however,areusuallymediatedfirstintheareasofteachingandschoolorganization,asnotedabove.Theseimprovementstaketimetoaffectpositivelystudentlearninggains,whichcurrentlyarebestmeasuredthroughstandardizedtests.Thus,studentperformancelevelsshouldbetrackedandevaluatedlongitudinallytogaugetheimpactofleadershipbothpriortoandduringaprincipal’stenure.Implementationexperts,however,suggestthatstudentachievementgainsasaresultofleaderactionswillnotbeapparentforatleast3-5yearsafterimprovementworkbegins(Fullan,2001).

EvaluationPlanningGuideAlignmenttoNationalAccreditationEvaluationExpectationsThisplannerisalignedtorecommendedevaluationoutcomesforprogramsseekingnationalaccreditationthroughtheCouncilfortheAccreditationofEducationalPrograms(CAEP).UnderCAEP,individualeducationalleadershippreparationprogramswithincollegesorschoolsofeducationarereviewedforrecognitionstatusbyaspecializedprofessionalassociation(SPA)usingnationallyrecognizedstandards.TheEducationalLeadershipConstituentCouncil(ELCC)istheCAEP-SPAforleadershippreparationandisgovernedbytheNationalPolicyBoardforEducationalAdministration(NPBEA–www.npbea.org).ELCCprovidesbothstandardsandguidelinestoprogramsforthepreparationofeducationalleaders,includingexpectationsforprogramevaluation.Together,these

TheINSPIRESuiteofSurveyshasbeendesignedtoprovidemeasuresformostoftheseevaluationcategories.ThesuiteoffoursurveysincludestheINSPIREPreparationProgramSurvey,theGraduateSurvey,theLeaderinPracticeSurvey,andthe360survey.Thesesurveyshavebeenfield-testedanddemonstratedstrongreliabilityandvalidity,andarewell-alignedwithnationaleducationalleadershipstandards.FormoreinformationonINSPIREseehttp://www.edleaderprep.org/??????

Page 10: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page8

standardsandguidelinesclarifyexpectationsagainstwhichindividualprogramscanbecompared,includingthekindsofdataandoutcomesforprogramimplementationandimpactassessments.

ELCCrequiresseventypesofassessmentsandrecommendsspecificevidence:

• statelicensureassessmentorothercontent-basedassessment;• assessmentofcontentknowledgeineducationalleadership,usingcomprehensive

examinations,essays,andcasestudies;• assessmentofabilitytodevelopsupervisoryplanforclassroom-basedinstruction,

suchasschoolimprovementplans,needsassessmentprojects,andfacultyinterventionplans;

• assessmentofinternship/clinicalpracticeusingfacultyevaluationsofcandidates’performances,internship/clinicalsitesupervisors’evaluationsofcandidates’performances,orcandidates’formativeandsummativelogsandreflections;

• assessmentofabilitytosupportstudentlearninganddevelopment,suchaspost-graduate360surveys,employersatisfactionsurveys,andcommunityfeedbacksurveysofcandidatesorgraduates;

• anassessmentofthecandidates’applicationofcontentknowledgeineducationalleadership(suchasactionresearchprojectsandportfoliotasks);and

• anassessmentofcandidates’abilitiesinorganizationalmanagementandcommunityrelations(suchas,school-basedstrategicplans,schoolsimulations,andschoolinterventionplans).

WhenreviewedbyELCC,programsareratedontheiruseandqualityoftheseseventypesofassessment.Qualityisdeterminedby:

• theextenttowhichtheassessmentdescriptionandscoringguidesarealignedtospecificELCCstandardelements;

• howthescoringguideisusedtomeasureprogress;• howaggregateddataarealignedtospecificELCCstandardsandtheassessment

scoringguide;and• whetherresultsshowbothareasofcandidatesuccessandprovideanimprovement

planforareasinwhichcandidatesarenotsuccessful.Toencouragethatevaluationdataareusedforprogramimprovementandimprovedgraduatepreparation,ELCCrequiresthatprogramsdescribehowtheirfaculty“areusingthedatafromassessmentstoimprovecandidateperformanceandtheprogram,asitrelatestocontentknowledge;pedagogicalandprofessionalknowledge,skills,anddispositions;andstudentlearning”(NationalCouncilforAccreditationofTeacherEducation,2008)(p.2).Thus,data

Page 11: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

collection,analysisandusemustbedocumentedandtheirrelationshiptoprogramdecisionsshouldbenoted.

EvaluationPlanningWorksheetThesecondcomponentoftheevaluationguideisaworksheetforprogramstocompleteintheirevaluationplanning,seeTable1below.Importantly,programofficialsshouldidentifyoneormoresourcesofevidenceforeachcategory.AblankformisprovidedasTable2tobeusedforprogramevaluationplanningpurposes.

Table1.EvaluationPlanningGuideWorksheet

COMPONENTS POSSIBLEMEASURESAndASSESSMENTS

DATASOURCE(s)

TIMELINE RelationshiptoCAEP/ELCCRequirements

PRE-CONDITIONS Admissioncriteria,guidelinesEnnrolleeprofile/descriptions,includingdemographics,prioreducational&leadershipexperience,etc

INSPIREPrepProgramsurveyEnrollmentdocumentationObservations/codedwithrubrics

Beforeoratbeginningofprogramstart

Documentationneeded

PROGRAMFEATURES

DescriptionofSpecificProgramElementsCandidatefeedback&assessmentofspecificProgramFeatures&OverallProgramQuality

INSPIREPrepProgramsurveyProgramDocumentationINSPIRE-GraduateSurvey

DuringtheProgramAtornearProgramCompletion

ELCCevaluationrequirement

FORMATIVELEARNING

PortfolioofprogramlearningactivitiesStandards-basedtasks&projects–withincourses&/oracrossprogramInterns’documentationoftheiraccomplishmentsforschoolsandstudentlearning

GradesProgramdevelopedassessmentrubricsOn-linemanagementprogram(e.g.LiveText,Chalk&Wire,other)

Throughout/atregularintervalsthroughouttheprogram

ELCCevaluationrequirement

SUMMATIVELEARNING

Stateornationalleadershipassessments

ETSPRAXISexamorother

Atorneartheendoftheprogram

Page 12: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page10

COMPONENTS POSSIBLEMEASURESAndASSESSMENTS

DATASOURCE(s)

TIMELINE RelationshiptoCAEP/ELCCRequirements

Self-reportsonlearningefficacybyleadershipareaPre-postassessmentofleadershipknowledgegains,usingaknowledgeassessmenttool

StandardizedassessmentsINSPIRE-GraduatesurveysectiononLearningOutcomesProgram-specificrubricorAssessmentcenterevaluation(e.g.NASSP)

AtorneartheendoftheprogramAtappropriateintervalsduringtheprogram

CAREERADVANCEMENTOUTCOMES

Whetheradvancedtoasupervisoryschoolleadershipposition(assistantprincipalorprincipal)Whetheradvancedtootherleadershippositions(districtornonsupervisory)Lengthoftimetoadministrativeplacementoradvancement

EmploymentdocumentationINSPIREGraduateSurvey&/orLeaderinPracticesurvey

Atregularintervalsafterprogramcompletion

ELCCevaluationrequirement

LEADERSHIPPRACTICES

Self-reportofeffectiveleadershippracticesPerceivedefficacyasschoolleaderSupervisorratingofprincipalleadershippracticesorefficacyTeacherratingofprincipalleadershippracticesorefficacy

INSPIRELeaderinPracticeSurveyStateorotherstandardizedPrincipalevaluationsystemINSPIRE-360,VAL-EdorotherINSPIRE-360,VAL-Edorother

Atregularintervalsafterprogramcompletion(e.g.1-3yearintervals)

STAFFANDSCHOOLPRACTICES

Distributedleadership/teacherroleinpolicymakingTeachercollaboration(PLCs)&sharedproblem-solving

INSPIRE360surveyDistrictclimatesurveys

Atregularintervalsafterprogramcompletion(e.g.1-3yearintervals)

Page 13: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

COMPONENTS POSSIBLEMEASURESAndASSESSMENTS

DATASOURCE(s)

TIMELINE RelationshiptoCAEP/ELCCRequirements

ChangesinschoolconditionsthatsupportstudentlearningChangesinhowstaffworktoimproveinstructionaleffectivenessDistrictSupport

STAFFANDSCHOOLEFFECTS

StudentengagementFamilyengagementCollectiveProfessionalEfficacyAcademicrigororpressofschoolReductioninschoolproblemsthatinterferewithlearningStaffattendanceTeacherRetention

INSPIRE360surveyDistrictannualreportsonstaffDistrictclimateorotherrelevantsurveys

Atregularintervalsafterprogramcompletion(e.g.1-3yearintervals)

STUDENTOUTCOMES

Reductioninstudentproblemsthatinterferewithlearning(student-related)StudentachievementStudentattendanceStudentRetentionStudentCompletion/Graduation

Districtassessmentsandreportsonstudentperformance

Trackannually,butlookforeffectslongitudinally(everythreeyears)inaschoolleaderposition

Page 14: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page12

Table2.EvaluationPlanningGuideBlankWorksheetforProgramUse

COMPONENTS POSSIBLEMEASURESANDASSESSMENTS

DATASOURCE TIMELINE

PRE-CONDITIONS PROGRAMFEATURES

FORMATIVELEARNING

SUMMATIVELEARNING

CAREERADVANCEMENTOUTCOMES

LEADERSHIPPRACTICES

STAFFANDSCHOOLPRACTICES

STAFFANDSCHOOLEFFECTS

STUDENTOUTCOMES

Page 15: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

ReferencesDarlingHammond,L.,Meyerson,D.,LaPointe,M.M.,&Orr,M.T.(2009).Preparingprincipalsfora

changingworld.SanFrancisco,CA:Jossey-Bass.Fullan,M.(2001).Leadinginacultureofchange.SanFrancisco:Jossey-Bass.Goldring,E.,Porter,A.C.,Murphy,J.,Elliot,S.N.,&Cravens,X.(2007).Assessinglearner-centered

leadership:Connectionstoresearch,professionalstandardsandcurrentpractices.Nashville,TN:VanderbiltUniversity.

Jackson,B.L.,&Kelley,C.(2002).Exceptionalandinnovativeprogramsineducationalleadership.EducationalAdministrationQuarterly,38(2),192-212.

Leithwood,K.,&Jantzi,D.(2008).Linkingleadershiptostudentlearning:Thecontributionsofleaderefficacy.Educationaladministrationquarterly,44(4),496-528.

NationalCouncilforAccreditationofTeacherEducation.(2008).Programreportsforthepreparationofeducationalleaders:NationalCouncilonAccrediationforTeacherEducation,http://www.ncate.org/ProgramStandards/ELCC/ELCCWebReport(SchoolBldgLevel)July1.doc).

Orr,M.T.,Jackson,K.,&Rorrer,A.(2009).FollowingupGraduates:DevelopmentoftheSchoolLeadershipPreparationandPracticeSurvey(SLPPS)andaSharedResearchProcess.UCEAReview.

Robinson,V.M.J.,Lloyd,C.A.,&Rowe,K.J.(2008).Theimpactofleadershiponstudentoutcomes:Ananalysisofthedifferentialeffectsofleadershiptypes.Educationaladministrationquarterly,44(5),635-674.

Weiss,C.H.(1998).Evaluation:Methodsforstudyingprogramsandpolicies(Seconded.).UpperSaddleRiver,NJ:PrenticeHall.

Page 16: Developing Evaluation Evidence REVISED3fl71l2qoj4l3y6ep2tqpwra.wpengine.netdna-cdn.com/wp... · 2018. 1. 10. · This publication was developed and produced by the National Center

NationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice Page14

TheNationalCenterfortheEvaluationofEducationalLeadershipPreparationandPractice

Highqualityleadershippreparationisessentialtoeducationalreformandimprovedstudentachievement.Tosupporthighqualityleadershippreparation,theUCEANationalCenterfortheEvaluationofEducationalLeadershipPreparationandPracticeprovides:

1. Surveyandevaluationresearchforprogrambenchmarkingandanalysisofprogramfeatures,graduatecareerandleadershippractices,andrelatedschoolandstudentoutcomes.

2. Asystematicprocessforcollectingandanalyzingstatedataondegreesandcertification,careeradvancement,andschoolprogressbygraduates.

3. Technicalassistanceandsupportforleadershippreparationprograms,includingregionaltrain-the-traineropportunitiestoincreaseevaluationcapacitylocally.

4. Asustainablesystemforevaluationresearchtosupportprogramimprovement.

5. Policyanalysisandpolicydevelopmentsupport.

TolearnmoreabouttheNationalCenterandtheservicesavailable,pleasevisitourwebsiteathttp://www.edleaderprep.orgTheUniversityCouncilforEducationalAdministration(UCEA)isaninternationalconsortiumofuniversitiesthatofferdoctoralprogramsineducationalleadershipandadministrationandaremarkedbyadistinguishingcommitmentandcapacitytoleadthefieldofeducationalleadershipandadministration.UCEAhasasinglestandardofexcellenceformembership:Superiorinstitutionalcommitmentandcapacitytoprovideleadershipfortheadvancementofeducationalleadershippreparation,scholarship,andpracticeconsistentwithUCEA'sestablishedmission.UCEA'smissionistoadvancethepreparationandpracticeofeducationalleadersforthebenefitofallchildrenandschools.UCEAfulfillsthispurposecollaborativelyby1)promoting,sponsoring,anddisseminatingresearchontheessentialproblemsofpractice,2)improvingthepreparationandprofessionaldevelopmentofschoolleadersandprofessors,and3)influencingpolicyandpracticethroughestablishingandfosteringcollaborativenetworks.TolearnmoreaboutUCEA,pleasevisitourwebsiteatwww.ucea.org