Assessment Series Part 3: Curriculum Mappingfor Program Assessment

J. DickinsonProvost’s Faculty Fellow for Assessment

Goalsfortoday’ssession:

• Introducecurriculummappingaspartoftheoverallassessmentplanningprocessanddiscussitsvalue

• Breakdownthestagesofdevelopingacurriculummapinto”doable”chunks

• Lookatandtryoutsomeworksheetsandothertoolstobegincurriculummapping

• Leavewithclearlyarticulated“Nextsteps”foryourprogram

Goalsofassessment

• Clearidentification/descriptionofprogramcharacteristicsandexpectationsforstudentachievement

• Systematiccollectionofdifferentkindsof(helpful)datatoevaluatestudentprogresstowards/achievementofthoseexpectations

• Useofanalyzeddatatoinformcurricularrevision

• Repeat

ProgramAssessmentisaDepartmentalEffort

• Atdifferentstagesoftheprocess,differentgroupsmayparticipate,ora’core’offacultymayfocusoncurriculumandprogramassessment

• Multipleperspectivesarenecessaryindevelopingoutcomesandassessingstudentprogresstowardsthoseoutcomes

Fewprogramsstartatzero

Eveniftheydonothaveclearlystated/updatedoutcomes,mostprogramshavebeengatheringand,tosomedegree,reviewingsomedataonstudentsuccessintheirprogramallalong:

• Facultyimpressions/frustrations• Studentsatisfaction/dissatisfaction/painpoints• Gradesorothercourse-levelassessmentsthatindicatestudentskilllevels• Anecdotalevidenceaboutstudentsuccessaftergraduation

StepsinProgramAssessment– Whereareyou?• Developprogram-leveloutcomeswithinput/draftsbyfaculty• Mapcurriculumtoidentifyplaceswherestudentslearn,practiceanddemonstratetheirmasteryoftheoutcomes• Gatheradditionalinformationaboutstudentprogresswithinthiscurriculum:

• Surveysofmajorsand/oralums• Facultysummariesofstudents’performanceonexpectedskills• Otherindicators(retentionofskillsfromprerequisitecourses;identifyingpredictorsofstudentssuccess,etc.)

• Usethisinitialinformationtodevelopaplanforassessmentofstudentprogresstowards/achievementoflearningoutcomes

Whatisacurriculummap?

Acurriculummapormatrixisatooltolinkelementsofaprogram(usuallyrequiredcoursesandexperiences)tolearningoutcomes.

Thegoalofcurriculummappingistoidentifyhowgraduation-levelmasteryoftheoutcomesisscaffoldedacrosstheprogramcurriculum,aswellaswherestudents’progressisassessedandfeedbackgiven

Whymapyourcurriculum?

• Createasharedunderstandingoftheprogram• Checkfor“drift”incoursegoals- theroleofcoursesinthecurriculummayhavechangedovertime• Doagapanalysistoseewhereoverlapsinemphasis,orgapsincoverage,mayslowstudentprogress• Ensurethatstudentsarereceivingsufficientfeedbackonkeyskillsandknowledgeastheydevelopthelevelofmasteryexpected• Identifypointsinthecurriculumwhereexistingassessmentscanprovideinsightsintostudentprogress

Mapformats

Thebasicform

Whilecurriculummapscantakemanyforms,mostarepresentedasachartthatlistslearningoutcomesononeaxisandkeycourses,experiencesorassessmentsalongtheother

CoursesandExperiences

ProgramLearningOutcomes

Applythescientificmethod DeveloplaboratorytechniquesDiagramandexplainmajor

cellularprocesses

Awarenessofcareersandjobopportunitiesinbiological

sciences

BIOL101

BIOL202

BIOL303

BIOL404

Other:Exitinterview

SampletemplatefromRIT

Fillinginthemap

Thechartisfilledinusingakey(usuallyletters)thatrecordinformationaboutwhatisbeingtaughtandthelevelofskillorknowledgestudentsaredemonstratinginthecourse.

CoursesandExperiences

ProgramLearningOutcomes

Applythescientificmethod DeveloplaboratorytechniquesDiagramandexplainmajor

cellularprocesses

Awarenessofcareersandjobopportunitiesinbiological

sciences

BIOL101 I I I

BIOL202 R R I

BIOL303 R M,A R

BIOL404 M,A M,A R

Other:Exitinterview A

SampletemplatefromRIT

Samplemapworksheet:UVMPhilosophy

LearningOutcomes

Coursesinwhichoutcomesareassessed

LO1:Readphilosophicalwritingclosely,discerningthestructureofphilosophicalargumentation

LO2:Accuratelyandcriticallyanalyzeargumentsandpositions

LO3:Writeclearandcoherentphilosophicalprose

Adifferentformat(andacomplexmap):

StepbystepHow-to’s

1. Identifytheaxes2. Populatetheaxes3. Chooseorcreateakey

a) Whichkeydoyourecommend?

4. Gatherdatafromfaculty5. Compile6. Alignment7. Usingwhatyouhavelearned

Firststep:Identifytheaxes

• Mostcurriculummapshaveoutcomesacrossthetop,andcourseslisteddowntheleft-handside;theseaxescanbereversed

• However,youcanalsousethismatrixapproachtoorganizemanykindsofdata,e.g.wheredifferentkindsofcourse-levelassessmentsoccur,orhowtypesoflearningexperiencesaredistributedacrossrequiredvs.electivecoursework

• Take5minutestolookatthesamplemapandthinkaboutwhatwouldbemostusefultomapforyourprogram

Step2:Populatetheaxes

• Decidehowmuchdetailyouwantintheoutcomesaxis.Forexample,ifyouhaveafewlearningoutcomeswithanumberofsub-components,youmaychoosetostaywiththelargeroutcomesorspliteachoutcomeintoitscomponents.

• Choosewhethertoincludeonlyrequiredcourses/experiences,orwhethertoalsomapoutelectivecourseworkorcourseworkforoptionaltracks.

• Let’sdiscusshowyouwouldpopulatetheaxes

Step3:Chooseorcreateakey• Therearealmostasmanysetsofdescriptorsutilizedincurriculummappingastherearebooksonassessment• Oneofthemostcommononesis:I/R/M/A• I=introduced;R=reinforced;M=mastery(graduationlevel);A=assessed

• Stassenetal.useI/E/U/A• Introduced/Emphasized/Utilized/ComprehensiveAssessment

• Otherkeysfocusmoreonstudentbehavior,e.g.I/P/D• Introduced/Practiced/Demonstrated

• And/orlevelexpectationsforprogressonanoutcomee.g.B/I/A• Beginning/Intermediate/Advanced

Alternative approach:

RSENR Core – level of instruction/learning

Step3a:Isthereakeyyourecommend?

Irecommendakeythatfitsthewayyourprogramisorganizedandyourimmediategoalsforthecurriculummappingprocess.Forexample:• Akeyfocusedonfacultybehaviors(e.g.IRMA)willhelpidentifywherefacultyareteachingaspectsoftheoutcomes• Akeyfocusedonstudentperformance(e.g.IPD)willhelpfocusonwherestudentsareaskedtodemonstrateprogressontheoutcomes• Akeyfocusedonlevel(e.g.B/I/A)willhelpidentifywhetherelementsoftheoutcomesaresufficientlyscaffoldedacrossthecurriculum

Whatifwewanttoknowallofthis?

• Curriculummappingcanbeanintensiveprocessthatmovesthroughseverallevels,oryoumayfindthatasingle‘pass’throughthecoursesisenough.

• Youcanhavefacultyfilloutseveralmatricesusingdifferentkeys

• Irecommendkeepingthematricesseparateatfirst,soyoucanseepatternsemergingthatarerelevanttoeachkey,beforecombiningthedata.

Step4:Gatherdatafrominstructors

• Somedatamaybeavailablefromsyllabi,howeverinstructorswillbeabletoquicklyidentifytheoutcomes/outcomecomponentscoveredintheircoursesaswellasassessmentsofstudentprogressonthoseoutcomes

• Forafirstpass,simplyaskinstructorstofillouttherowsofthematrixfortheclassestheyteach,usingONLYthekeythatyouhavechosen– nonotes,asterisksorcaveats

• Dependingonyourdepartment/curriculum,thiscanbedonetogetherorindividually

Giveitatry

• Listthecoursesyouteach,orcoursesinyourprogramthatyouarefamiliarwith

• Usingthekeyyouthinkismosthelpful,tryfillinginthematrixforthosecourses

• Nowthatyouhavetriedit,whatdifficultiesdoyouanticipatewhenyouaskyourfacultytocompleteamatrix?

Step5:Compile

Ifindividualinstructorshavefilledoutthematrix,combinetheirresultsintoasinglematrix,notinganyareasofvariationacrossinstructorsforasinglecourse

Beforediscussingalignment,itmaybeproductivetohaveadiscussionaboutthemappingprocess

Step6:Alignment• Iscoursecontent/workalignedwiththeoutcomes?

• Arethereanygaps?Overlaps?Ifso,aretheseproblematic?

• Isstudentprogressappropriatelyscaffoldedacrossthecurriculum?

• Wherearestudentsassessedontheirprogress?

• Aretheredirectorindirectcourse-levelorprogram-levelassessmentsinplacethatcanbeusedtocheckalignment?

Step7:Usewhatyoulearned

• Thecurriculummappingprocessmayhighlightsomeareasforimmediatechange(e.g.agapinyourcoverageofoutcomes)

• Themapcanalsoserveasthebasisfordevelopinganassessmentplan.• Arestudentsactuallyworkingattheexpectedlevelintheseclasses?• Whichcourse-levelassessmentscanprovideinformationonstudentprogress?• Atwhatpointintheprogramwillstudentsachievegraduation-levelachievementoftheoutcomes?

Questions?

J. Dickinson: jadickin@uvm.edu