phd in counseling & supervision annual report 2018-2019
TRANSCRIPT
CACREPANNUALREPORT
AcademicDegreeProgram:PhDinCounseling&Supervision
DepartmentHead:RobinAnderson
AssessmentCoordinator:DebbieC.Sturm
ThePh.D.inCounselingandSupervisionisaprogramcommittedtotrainingcounseloreducators,supervisors,andadvancedpractitionerswithexpertiseinthetheory,researchandpracticeofcounseling.Ourmissionistocreatedynamicandinnovativeleaderswhowillserveascatalystsforprogressinaddressingthetremendouscounselingneedsofindividuals,families,institutions,andcommunities.Thestudents,facultyandstaffmembersofthisprogramvaryinage,class,gender,ethnicity,race,religion,sexualorientation,andphysicalabilities.Puttingourprinciplesintopractice,thesediverseindividualscreatealearningcommunityinwhichstudentsthrivepersonally,growprofessionallyandshareacommoncommitmenttocounseling.Wechallengeourstudentstocontinuetheirlife-longjourneysofexploringpossibilities,refiningskillsandmaintainingtheirprofessionalvitalitythroughouttheircareersascounselors.Weencouragethemtosupportoneanotherintheformidabletaskofbecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining.
ThePh.D.inCounselingandSupervisionwasapprovedbytheStateCouncilofHigherEducationinVirginia(SCHEV)andimplementedin2011withacohortoffivedegree-seekingstudents.Thecurriculumfocusesoncounseloreducation,supervisionstrategies,leadershipskills,counselingservices,andresearchmethods.Eligibleapplicantsarepracticingcounselorswhohavecompletedanadvanceddegree(M.A.,M.S.orEd.S.)fromacounselingmaster’sdegreeprogramand60credithoursofgraduatecounselingcourseworknecessaryforlicensureasaprofessionalcounselor,withpreferencegiventograduatesofCACREPaccreditedprograms.Theemphasisoncrisiscounselingandemergencyservicesisalsoauniquefeatureoftheprogram.
Duringthe2018-2018academicyear,16studentshavebeenenrolledinthePh.D.inCounselingandSupervisionProgram.Sincetheimplementationoftheprogram,eightdoctorateshavebeenawarded.Duringthepasttwoyears,weadmittedatotalof7students.Ourfull-timestudentsareexpectedtocompletetheprograminthreeyearsandourpart-timestudentsplantocompletetheprograminfiveyears.Duringthefirstfouryearsoftheprogram,ourcombinedcompletionrateforfull-timeandpart-timestudentsinthePh.D.inCounselingandSupervisionProgramhasbeen89%.Allofouralumnireportthattheyarelicensedandworkingeitherfull-timeorpart-timeincounseloreducation,supervision,andcounselingpractice.
ThePhDinCounseling&Supervisionincludesa48-hourcurriculumbasedonthe2016standardsestablishedbytheCouncilforAccreditationofCounselingandRelatedEducationalPrograms(CACREP),whichrequiresdemonstrationofmasteryacrossfivecorecounselingareasaswellasatleastoneadditionalareaspecificallyrelatedtothespecificmissionorvaluesoftheprogram.Inourcase,theadditionalareacurrentlyindevelopmentisCrisis,Emergency&Disasterresponse.Counselingprogramsarealsoaskedtoincludeprogram-specificstandardsthatareconsonantwiththecultureandexpectationsofthatspecificprogramaswellasaprofessionaldispositionsmetricintheformofregularprogressreviews.Forthesakeofclarityandconsistency,theprogramspecificexpectationshavebeenmapped,whereappropriatetotheCACREPstandards(seeAPTElement1A,below).Inadditiontootherconsiderations,CACREPrequiresevaluationofKeyPerformanceIndicators(KPIs)ofstudentlearningineachoftheabovementionedareasviamultiplemeasuresandovermultiplepointsoftime.Thespreadsheetdocumentingourcurrentplanforthisevaluationisattached,withspecificinformationrelevanttothisAPTincludedbelow.ThisassessmentplanwasfirstimplementedbeginninginFall2018intandemwithourprocessofsubmittingtheSelf-StudytoCACREP.Wearecontinuallycollectingdatafromconstituentstohelpinformourcurriculum,andininformalconsultationwithinternalandexternalstakeholders,weareaddingaseriesofworkshopsdesignedtoenhanceourstudents’preparationforemploymentasCounselorEducatorsoradvancedsupervisorsinclinicalsetting.
TheAssessmentPlanisintendedtoprovideaframeworkofprogramevaluationandassessmentthatalignswithJamesMadisonUniversity’s(JMU)counselingprograms’missionandidentifiedstudentlearningobjectives.ThecounseloreducationfacultyatJMUarecommittedtoprogramqualityandcontinuousimprovement.Thus,theprocessofevaluationandassessmentisintegratedinallcomponentsofgraduatetrainingandprogramplanning.Thisincludesevaluationprocessesforadmissionsdecisions,studentmatriculation,courseeffectiveness,facultyperformance,supervisorperformance,andemployerandalumniexperiences.PertheAssessmentPlan,assessmentandevaluationwithintheJMUcounselingprogramsarealignedwiththeCouncilfortheAccreditationofCounselingandRelatedEducationalPrograms(CACREP)standards.JMUcounselingfacultymeettwiceamonthtodiscussCACREPstandards,reflectoninfusingthesestandardsintoprogramandstudentevaluationassessmentprocesses,andcollectingthisdataacrossprograms.Allcollecteddataareanalyzedbyprogramfacultyandreportedtostakeholders(e.g.,administration,accreditationteams,alumni,employers,potentialstudents)asrelevant.StakeholdersprovideprogramandcurricularfeedbacktocontinuallyimprovethecounselingtrainingprogramsatJMU.Theprocessofevaluationandassessmentisdescribedinfurtherdetailbelow.EvaluationoftheCounselingProgramsApproach
TheAssessmentPlanwasdesignedbyfacultyasaformalevaluationprocessthatdocumentstheirempiricallybasedapproachforsystematicallyevaluatingprogramobjectives,keyperformanceindicators(KPIs),studentlearningwithanemphasisonacademicperformanceandprofessionaldisposition,andfaculty/supervisorperformance.TheAssessmentPlanincludesadescriptionofthedatacollected,themethodforhowthedatawascollected,adescriptiononhowthedatawasanalyzed,howthedatainformedcurriculum,,andprogramimprovementsacrossallthreecounselingprograms.Thevisualchartbelowdescribesthefourcomponentsofthisevaluationplan(A.1.ProgramEvaluation;A.2.StudentAssessment;A.3.EvaluationofFaculty;andA.4.EvaluationofSupervisors)andprovidesinformationonthetypesofassessmentsutilizedtocollectevaluativedata.
StudentLearningObjectives(APTElement1A&1B)Studentlearningobjectives(SLOs)arestatementsindicatingwhatstudentsshouldknow,think,ordoasaresultofparticipatinginanacademicdegreeprogram.SLOsshouldbestudent-centeredandbewrittenclearlywithprecise,measureableverbs.ThereisnosetnumberofSLOsrequiredfortheAPT.
StudentLearningObjectiveAsaresultofparticipatingintheDoctoralPrograminCounseling&Supervision,studentsgraduatingwithanPhDinCounseling&Supervisionwill:
KPI1Counseling:Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.Takeresponsibilityforembodyingandpracticingprofessionalandethicalstandardsatthedegreeofmasterybyeffectivelydemonstratingcounselingskillsincludingtheabilitytoapplycounselingtheoriesandevidence-basedmethods,evaluateclientprogress,anddemonstrateethicalandculturally
competentcounselingpractice.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.
AlignedwithCACREPCounselorEducationandSupervisionStandards:B.1.a-f1. COUNSELING
1. scholarlyexaminationoftheoriesrelevanttocounseling2. integrationoftheoriesrelevanttocounseling3. conceptualizationofclientsfrommultipletheoreticalperspectives4. evidence-basedcounselingpractices5. methodsforevaluatingcounselingeffectiveness6. ethicalandculturallyrelevantcounselinginmultiplesettings
KPI2Supervision:Knowledgeandskillsforeffectiveclinicalsupervision.Takeresponsibilityfordevelopingapersonal,ethical,research-basedandculturallyrelevantidentityasasupervisorbydemonstratingknowledgeoftheoriesandmodelsofclinicalsupervisionandtheskillsnecessarytoprovideeffectiveclinicalsupervision.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.
AlignedwithCACREPCounselorEducationandSupervisionStandards:B.2.a-k
2. SUPERVISION1. purposesofclinicalsupervision2. theoreticalframeworksandmodelsofclinicalsupervision3. rolesandrelationshipsrelatedtoclinicalsupervision4. skillsofclinicalsupervision5. opportunitiesfordevelopingapersonalstyleofclinicalsupervision6. assessmentofsupervisees’developmentallevelandotherrelevantcharacteristics7. modalitiesofclinicalsupervisionandtheuseoftechnology8. administrativeproceduresandresponsibilitiesrelatedtoclinicalsupervision9. evaluation,remediation,andgatekeepinginclinicalsupervision10. legalandethicalissuesandresponsibilitiesinclinicalsupervision11. culturallyrelevantstrategiesforconductingclinicalsupervision
KPI3Teaching:Knowledgeandskillsrelevanttoeffectiveteaching.Takeresponsibilityfortrainingascounseloreducatorswithexpertiseinthetheory,researchandpracticeofcounselingasevidencedbydemonstratingaknowledgeofpedagogyandskillsrelevanttoteachingincludinginstructionaldevelopment,implementation,andevaluationmethodsincounseloreducation.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.
AlignedwithCACREPCounselorEducationandSupervisionStandards:B.3.a-i.
3. TEACHING1. rolesandresponsibilitiesrelatedtoeducatingcounselors2. pedagogyandteachingmethodsrelevanttocounseloreducation3. modelsofadultdevelopmentandlearning4. instructionalandcurriculumdesign,delivery,andevaluationmethodsrelevant
tocounseloreducation
5. effectiveapproachesforonlineinstruction6. screening,remediation,andgatekeepingfunctionsrelevanttoteaching7. assessmentoflearning8. ethicalandculturallyrelevantstrategiesusedincounselorpreparation9. theroleofmentoringincounseloreducation
KPI4Research&Scholarship:Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.Weencouragestudentstosupportoneanotherprofessionallyinthetaskofbecomingleaderswhoadvancethecounselingprofessionthroughresearchandinnovationasevidencedbydemonstratingtheknowledgeandskillsnecessarytoengageinresearchandscholarship,includingquantitativeandqualitativeresearch,thatadvancedthefieldofcounseling,counseloreducation,andsupervision.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.AlignedwithCACREPCounselorEducationandSupervisionStandards:B.4.a-l
4. RESEARCHANDSCHOLARSHIP1. researchdesignsappropriatetoquantitativeandqualitativeresearchquestions2. univariateandmultivariateresearchdesignsanddataanalysismethods3. qualitativedesignsandapproachestoqualitativedataanalysis4. emergentresearchpracticesandprocesses5. modelsandmethodsofinstrumentdesign6. modelsandmethodsofprogramevaluation7. researchquestionsappropriateforprofessionalresearchandpublication8. professionalwritingforjournalandnewsletterpublication9. professionalconferenceproposalpreparation10. designandevaluationofresearchproposalsforahumansubjects/institutionalreview
boardreview11. grantproposalsandothersourcesoffunding12. ethicalandculturallyrelevantstrategiesforconductingresearch
KPI5Leadership&Advocacy:Knowledgeandskillsforeffectiveleadershipandadvocacy.Weencouragestudentstosupportoneanotherasprofessionalsinthetaskofbecomingleaderswhoadvancethecounselingprofessionthroughservice,innovation,andadvocacyasevidencedbydemonstrationofsufficientknowledgeandskillstobeeffectiveinadvocacyandleadershipinthecounselingprofession.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.AlignedwithCACREPCounselorEducationandSupervisionStandards:B.5.a-l.
5.LEADERSHIPANDADVOCACY1. theoriesandskillsofleadership2. leadershipandleadershipdevelopmentinprofessionalorganizations3. leadershipincounseloreducationprograms4. knowledgeofaccreditationstandardsandprocesses5. leadership,management,andadministrationincounselingorganizationsandother
institutions
6. leadershiprolesandstrategiesforrespondingtocrisesanddisasters7. strategiesofleadershipinconsultation8. currenttopicalandpoliticalissuesincounselingandhowthoseissuesaffectthedaily
workofcounselorsandthecounselingprofession9. roleofcounselorsandcounseloreducatorsadvocatingonbehalfoftheprofessionand
professionalidentity10. modelsandcompetenciesforadvocatingforclientsattheindividual,system,andpolicy
levels11. strategiesofleadershipinrelationtocurrentmulticulturalandsocialjusticeissues12. ethicalandculturallyrelevantleadershipandadvocacypractices
KPI6Crisis&DisasterResponse:Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.Weencouragestudentstoembracetrainingasadvancedpractitionerswithexpertiseinthetheory,researchandpracticeofcounselingasitrelatestocrisis,emergencyanddisasterresponseasevidencedbydemonstrationofsufficientknowledgeandskillsoftheimpactofcrisisanddisasteronclientsaswellasknowledgeandskillinrespondingtocrisisanddisastersclinicallyandsystemically.(NOTE:Thisisanareawearestilldeveloping.AtthispointwehaveoneclassandareworkingtoaddtothatwithelectivesandKPIsinothercourses.)
AlignedwithCACREPCounselorEducationandSupervisionStandards:
B.5.f.leadershiprolesandstrategiesforrespondingtocrisesanddisastersB.5.g.strategiesofleadershipinconsultationB.5.h.currenttopicalandpoliticalissuesincounselingandhowthoseissuesaffectthedailyworkofcounselorsandthecounselingprofession
KPI7KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.
AlignedwithCACREPCounselorEducationandSupervisionStandards4.G.
Thecounseloreducationprogramfacultysystematicallyassesseseachstudent’sprofessionaldispositionsthroughouttheprogram.Theassessmentprocessincludesthefollowing:(1)identificationofkeyprofessionaldispositions,(2)measurementofstudentprofessionaldispositionsovermultiplepointsintime,and(3)revieworanalysisofdata.
Course/learningExperiences(APTElement2)IfstudentsareexpectedtomeetthestatedSLOs,theymustbeprovidedwithlearningopportunitiestoassisttheminmeetingtheSLO.LearningopportunitiesshouldbeclearlylinkedtoSLOs,andallSLOsshouldbecoveredbyatleastonelearningopportunity.Thisisoftenreferredtoasprogramtheory.Twooptionsforacurriculummapareprovidedbelow.Youmaychooseeitheroptionforpresentingyourcurriculummap.
Studentlearningobjective Courses/Experiencesmappedtotheobjective
Introductory Practice MasteryKPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper
PSYC882Counselingpracticumperformanceassessment
PSYC892CounselingInternshipperformanceassessmentComprehensiveExam:CounselingEssayandCasePresentation
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignment
PSYC892SupervisioninternshipSuperviseeConceptualization
ComprehensiveExam:SupervisionEssayandCasePresentation
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTraining
PSYC892SyllabusandTeachingPhilosophy
PSYC892TeachingPerformanceEvaluationComprehensiveExam:TeachingEssayandCasePresentation
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC605FinalExamPSYC840ResearchDesignProposals
PSYC881LiteratureReviewPSYC881ResearchAgenda
ProposalDissertationDissertation
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignment
PSYC810Assignment:EducationSession
ComprehensiveExam:Leadership&AdvocacyEssayandDemontrationofWork
KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.
PSYC866CrisisPresentation TBD(Thiselementisstillbeingdeveloped).
TBD(Thiselementisstillbeingdeveloped).
KPI7.KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthical
AssessedattheconclusionofeachsemesterviatheProgressReviewFormandFacultyReviewProcess.AlsoCVtoevidencetheir
AssessedattheconclusionofeachsemesterviatheProgressReviewFormandFaculty
AssessedattheconclusionofeachsemesterviatheProgressReviewForm
BehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.
professionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining
ReviewProcess.AlsoCVtoevidencetheirprofessionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining
andFacultyReviewProcess.AlsoCVtoevidencetheirprofessionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining
*Note:forthistable,programsmayplacean“X”intotheappropriateintersectingboxestoindicatewhichcourse/learningexperiencesmaptowhichobjective(s).Foreithertable,programsmayspecifycoverageoftheobjectivebycourse/learningexperiencesbyplacinga“1”,“2”,or“3,”etc.,intotheappropriateboxes,wherea1indicatesminimalcoverageandhigherscoresindicatemorecoverage.AssessmentMeasures(APTElement3A&3B)ToobtainresultsthatareusefulforevaluatingwhetherstudentsmetthestatedSLOs,instrumentsmustbeselectedtoelicitthedesiredknowledge,skills,orattitudesfromstudents.AllSLOsshouldbemeasuredbyatleastoneinstrument.Moreover,toobtainthestrongestevidenceofstudentlearning,SLOsshouldbemeasuredbyadirectmeasureofstudentlearning.
Objective DescriptionofInstrumentusedtoassessobjective
Direct/Indirect
1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(RubriconCanvas)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam
I-assessmentisbasedonwrittenproductD–studentisobservedD–studentisobservedI-assessmentisbasedonwrittenproduct
2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualizationComprehensiveExam
AllthreeinvolvebothD&I:D–studentisobservedI-assessmentisbasedonwrittenproduct
3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTraining(RubriconCanvas)PSYC892SyllabusandTeachingPhilosophyPSYC892TeachingPerformanceEvaluationComprehensiveExam
Allfourinvolveawrittenproduct(I)andsomedegreeofobservation(D):I-assessmentisbasedonwrittenproductD–studentisobserved
4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC840ResearchDesignProposalsPSYC609ResearchProposalPSYC605FinalExamPSYC881LiteratureReviewPSYC881ResearchAgendaProposalDissertationDissertation
I-assessmentsarebasedonwrittenproductI-assessmentisbasedonwrittenproductI-assessmentisbasedonwrittenproductANDD–studentisobserved
5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignment(RubriconCanvas)PSYC810Assignment:EducationSession(RubricforwrittenandobservedelementsonCanvas)ComprehensiveExam
I-assessmentisbasedonwrittenproductI-assessmentisbasedonwrittenproductD–studentisobservedD–studentisobservedI-assessmentisbasedonwrittenproduct
6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.
PSYC866CrisisPresentation I-assessmentisbasedonwrittenproductANDD–studentisobserved
7.KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.
ProgressReviews&CVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining
I-assessmentisbasedonwrittenproductANDD–studentisobserved
DesiredResults(APTElement3C)Toaidininterpretationofresults,itishelpfultodeterminearesultthatprogramshopestudentswillmeet(i.e.80%ofstudentswillpassanexam,orstudentswillincreasebyatleast10pointsfromthebeginningoftheprogramtotheendoftheprogram).Ideally,programswillhaveadesiredresultforeachobjectiveandwilljustifytheirdesiredresultbasedonexternalresearch,facultyconsensus,previousyears’results,etc.
Objective Instrument Desiredresult Justificationfordesiredresult
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam
90%orgreaterratingsof2inordertobeconsideredtohavepassedthiselement.100%passresultinordertopromotetointernship100%passresultinordertosuccessfullycompleteinternshipandbeeligibleforcomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualizationComprehensiveExam
90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetosupervisioninternship90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTrainingPSYC892SyllabusandTeachingPhilosophyPSYC892TeachingPerformanceEvaluationComprehensiveExam
90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetosupervisioninternship90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC840ResearchDesignProposalsPSYC609ResearchProposalPSYC605FinalExamPSYC881LiteratureReviewPSYC881ResearchAgendaProposalDissertation
90%ofevaluationscoresindicatelevel2(mastery)90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullyproposedfordissertationandsuccessfullydefendeddissertation.By“2ndattempt”,werefertosituationsinwhichstudentshavesignificantfeedbacktoaddressbeforeconsideredpassing.
ThespecificconsiderationscoveredintheresearchcoursesseriesaredesignedtocorrelatewithCACREPresearchknowledgeandskillsandaredesigned/measuredbasedontheexpertiseoftheA&Mfacultywhoteachthosecourses.Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignmentPSYC810Assignment:EducationSessionComprehensiveExam
90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.
KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.
PSYC866CrisisPresentation 90%ofallevaluationsatlevel2,indicatingmastery
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimately
tosuggestthestudentispreparedforpost-graduationpractice.
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).
ProgressReviews(form/rubric)¤tCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining
100%satisfactoryafteranynecessaryremediationplan
Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.
DataCollection(APTElement3D)Sounddatacollectionproceduresareintegralforobtaininghighqualityresults.Datacollectionconsiderationsincludewhichstudentsweresampled,howmanystudentsweresampled,whetherthestudentswererepresentativeofthestudentstowhichinferenceswillbemade,whetherdatawerecollectedatonetimepointormultipletimepoints,whetherstudentsweremotivatedtogivebesteffortontheassessments,etc.Datacollectionproceduresmaydifferbasedonwhetherselected-responseorperformanceassessmentsareadministered.Forexample,ifaperformanceassessmentisadministered,itmayalsobeusefultoincludemultipleratersandratertrainingpriortoscoringthestudentartifacts.
Objective Instrument Studentssampled Samplesize Timepoint(s) Motivation
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(Rubric)
Allstudents
2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)
PSYC860:1styear/1stsemester
Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,
PSYC882Counselingpracticumperformanceassessment
PSYC882:1styear/2ndsemester
PSYC892CounselingInternshipperformanceassessment
PSYC892:2ndyear/1stsemester
andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.
ComprehensiveExam(CE)Rubric
CE:Aftercompleting2ndyear
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignment
Allstudents
2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)
PSYC853:1styear/2ndsemester
Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.
PSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)
PSYC892:2ndyear/1stsemester
ComprehensiveExam(CE)Rubric
CE:Aftercompleting2ndyear
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTrainingRubric
Allstudents
2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)
PSYC863:1styear/1stsemester
Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.
PSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessment
PSYC892:2ndyear/1stsemesterand/or2ndsemester
ComprehensiveExam(CE)Rubric
CE:Aftercompleting2ndyear
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC605FinalExam
Allstudents
2(1styearcohort)2(2ndyearcohort)2from2ndyearcohortforproposalsand3from3rdyearcohortfordissertations
PSYC605:1styear/1stsemester
Thisinstrumentincludeoneevaluationfortheidentifiedthatcontributestotheoverallevaluationofarequiredcourse.StudentsareusuallyfairlymotivatedtoearnanAinthesecourses.Thisistruefor840,609,605,and881.WehaverubricsforDissertationProposalandDissertation.Allelementsmustbesuccessfullypassedinordertobeconsideredcompleteandstudentsarehighlymotivatedtobesuccessfulinthisprocess.
PSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)
PSYC840/609:2ndyear/1stsemesterand/or2ndsemesterPSYC881:2ndyear/2ndsemester
ProposalandDissertation(Rubrics)
Aftercompleting2ndyear
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubric
Allstudents
2(1styearcohort)2(2ndyearcohort)3from2ndyearcohortforproposals
PSYC8621stsummerPSYC8102ndsummer
Thisinstrumentsincludeoneevaluationfortheidentifiedthatcontributestotheoverallevaluationofa
ComprehensiveExamRubric
Aftercompleting2ndyear
requiredcourse.StudentsareusuallyfairlymotivatedtoearnanAinthesecourses.Studentswillnotprogressintheprograminsequenceswithoutcompletingcomprehensiveexamssuccessfully.Thustheyarehighlymotivated.
KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.
PSYC866CrisisPresentation
Allstudents
2(1styearcohort)
PSYC866hasbeeninthe1stsummerbutisbeingmovedtothe1stfalleffectiveFall2018.
Thisinstrumentisoneindicatorconsideredinthecourseevaluation.TheyaremotivatedtoearnanAinthecourse.
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).
ProgressReviews(form/rubric)¤tCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining
Allstudents
2(1styearcohort)2(2ndyearcohort)3from3rdyearcohort1from4th+year
Eachsemester Progressreviewsarecompletedbyfacultyeachsemesterandaddresscounselingdispositions.Ideally,studentsareintrinsicallymotivatedthroughtheirinterestinthefieldtoexhibittheexpecteddispositionsacrossacademicandexperientialdomains.
ValidityEvidence(APTElement3E)Validityevidencecreatesastrongercaseforeventualinferencestobemadefromscores.Withoutvalidityevidence,itisdifficulttodeterminewhetherassessmentsaretrulymeasuringtheintendedknowledge,skills,andabilitiesprogramshavedeemedimportantforstudents.Importantvalidityevidenceincludes,butisnotlimitedto,reliabilityestimates(e.g.Cronbach’salphaorinterraterreliability),correlationswithothervariables(e.g.,professionalcertificationexamresultsorcoursegrades),andfaculty/expertevaluationofassessmentcontent.Theprogramfacultymemberstakeahermeneuticapproachtocomprehensiveexamandotherstudentevaluationratingsandthereforegroupconsensusisrequiredpriortothereportingoffinalratings.Forthisreasonwedonotcalculateinter-raterreliability.Facevalidityisassumedwithsite-supervisoranduniversity-supervisorevaluationsofstudentsinpracticumandinternship.ThisisduetothefactthatevaluationitemsaredrawnfromtheCACREPDoctoralCounselorEducationandSupervisionstandards(beginningFall2016),whicharethestandardsbywhichourprogramisevaluatedandaccredited.Further,theCACREPstandardsarethebasisforthedesignofour
coursesandourstudents’evaluationsacrossthecurriculum.Constructvalidityappliestothestandards,asexpertsinthefielddeterminetheCACREPaccreditationstandards.Constructvalidityisalsomaintainedasfacultyexpertiseresultsindeepknowledgeofthefieldanditsassociatedstandards.Withsomeexceptions,notedbelow,thefacultywhoteachcoursesassociatedwiththeobjectivesidentifiedherearecorefaculty,whoarelicensedmentalhealthproviderswithdoctoralleveltraininginourfieldofexpertise.Corefacultymembersteachcoursesthatarealignedwiththeirprofessionalexperienceandexpertiseandthereforehavesufficienttrainingandcompetencetoassesslevelsofstudentmastery.Incasesinwhichcorefacultymembersarenotavailabletoteachthesecourses,weemployadjunctfacultymemberswhoarespecificallychosenfortheirtrainingandexperiencerelatedtothecoursecontent.Further,allcorefacultymembersgatherattheendofeachsemestertoevaluateeverystudent’sprofessionaldispositionsandourkeyCounselorEducationandSupervisioncompetenciesandprofessionaldispositionsviaourProgressReviewprocess.Ourcombinedevaluationsassistusindeterminingthevalidityofthespecificmeasuresidentifiedinthetablebelow.
Objective Instrument InstrumentEvaluator Progress
reviewrelevance?
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric
PSYC860AssignmentRubricinCanvasPSYC882PracticumPerformanceAssessmentPSYC892InternshipPerformanceAssessment(Counseling)ComprehensiveExamRubric(CounselingCompetenciesSectionI)
PSYC860CoreFacultyMemberPSYC882CoreFacultyMemberPSYC892CoreFacultyMemberCECommitteeof2CoreFacultyMembers
Yes
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric
PSYC853AssignmentRubricinCanvasPSYC892InternshipPerformanceAssessment(Counseling)ComprehensiveExamRubric(SupervisionCompetenciesSectionII)
PSYC853CoreFacultyMemberPSYC892CoreFacultyMemberCECommitteeof2CoreFacultyMembers
Yes
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric
PSYC863AssignmentRubricinCanvasPSYC892AssignmentRubricinCanvasPSYC892InternshipPerformanceAssessment(Teaching)ComprehensiveExamRubric(SupervisionCompetenciesSectionIII)
PSYC863CorefacultymemberPSYC892CoreFacultyMember&SiteSupervisorCECommitteeof2CoreFacultyMembers
Yes.
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)
PSYC605AssignmentRubricinCanvasPSYC840AssignmentRubricinCanvasPSYC609AssignmentRubricinCanvasPSYC881AssignmentRubricsinCanvasDissertationProposalRubric(partofComprehensiveExamRubric)DissertationDefenseRubric
PSYC605CoreA&MFacultyPSYC840&609DeptofGraduatePsychologyFacultyPSYC881CoreFamilyMembersCommitteeof2CoreFacultyMembers
Yes
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric
PSYC862AssignmentRubricinCanvasPSYC810AssignmentRubricinCanvasComprehensiveExamRubric(Section5forLeadership&Advocacy)
PSYC862&810CoreFacultyMembers2CoreFacultyMembers
Yes
KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.
PSYC866CrisisPresentation PSYC866AssignmentRubricinCanvas
CoreFacultyMembers
Yes
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).
ProgressReviews(form/rubric)¤tCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining
ProgressReviewDocument(Rubric)
Studentself-evaluation;ReviewbyallCoreFacultyMembers;FeedbackfromAdvisor
Yes
Results(APTElement4A&4B)Resultsareusedtoconveytostakeholdershowwellstudentsmetthestatedstudentlearningobjectives.Thus,resultsshouldbeclearlypresentedinrelationtothestudentlearningobjectives.Often,historicalresultsprovidecontextforthecurrentyear’sresultsandprovideinsightaboutstudentlearningtrends.
HistoricalResults
Objective Instrument Priorresults Priorresults Thisyear’sresults
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric
50%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
50%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
50%passatPracticelevel75%passatPracticelevel100%passatMasterylevel
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevelon1stattempt100%passatMasterylevel
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(100%on1stattempt)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)
KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.
PSYC866CrisisPresentation 100%ofallevaluationsatlevel2practicelevel
100%ofallevaluationsatlevel2practicelevel
100%ofallevaluationsatlevel2practicelevel
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounseling
ProgressReviews(form/rubric)¤tCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,
80%satisfactoryafteranynecessaryremediationplan
80%satisfactoryafteranynecessaryremediationplan
80%satisfactoryafteranynecessaryremediationplan(onepersonenteredremediationand
ProgramsStudentProgressReviews).
innovation,advocacy,andtraining
istakingaleaveofabsense).
Results(APTElement4A&4B)Inadditiontopresentingresults,itisimportanttointerpretwhattheresultsmean,ofteninthecontextofstudentlearningobjectives.Interpretationsshouldmakereferencetothespecifieddesiredresults,aswellascurricular/pedagogicalchanges.
Interpretations
Objective Instrument ActualResult DesiredResult
Interpretation
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric
100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel
100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt
Wefeelconfidentinourabilitytohelpstudentslearnthecorecompetenciesassociatedwithknowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.Wealsorelyonthecombinationofdataobtainedfromboththecomprehensiveexamandtheinternshipexperiencestomakeourdeterminationsregardingstudentlearninginthisarea.Afterall,theperformanceincounselinginternshiphasadirectrelationshiptothelearningthatoccurswithinthisKPI.
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric
59%passatPracticelevel90%passatPracticelevel100%passatMasterylevel
50%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt
Studentsthisyearpresentedacuriosityandmotivationregardingthissubjectmatterthatenabledthemtoprogressfurtherthansomepreviousclasses.GiventheinterconnectednatureofthePhDprogramlearning,astudentwhostruggledincounselingpracticumwillalsolikelyexperienceramnificationswithregardtoreadinessforsupervisioninternship.
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric
100%passatPracticelevel100%passatPracticelevel80%passatPracticelevelon1stattempt100%passatMasterylevel
100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt
Therequirementsforteachingareassessedatmultiplepointsandreadinesstomovefromthetheoryclasstoactualclassroomteachingrequiressufficientmasteryfromtheverybeginning.
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)
100%ofevaluationscoresindicatepassingatthePracticelevel
100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionof
Studentsweregivenmultipleopportunitieswithinwhichtolearnresearchmethodologyintheoryandinpractice.While
PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)
95%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)
secondsemester100%passresultby2ndattempt
somestruggleinthefirstfewcourses,theA&Msupportsystemaswellasmorecounseling-specificlattercourses(application)havehelpedwithaddressingthemasteryneededtodissertationproposalandfinaldissertation.
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)
100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)
Studentstypicallyscoreinourdesiredrangeregardingthisdomain.Wedoneedtodeterminewaystobringthetheoryand“proposed”advocacytomoreactivepracticeinthisarea.Talkingaboutversusdoingisthegoalforourstudentsandrevisitingtheassessmenttoolsisontheagenda.
KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.
PSYC866CrisisPresentation 100%ofallevaluationsatlevel2practicelevel
100%ofevaluationscoresindicatepassingatthePracticelevel
Thisisanareawehaveidentifiedasaspecialtyforourprogram;however,wecurrentlyonlyofferonecourseandassessatonepoint.Weareengagedindiscussionaboutwayswecansolidifyitasa
truespecialtyareawithinthecurrentprogramstructure.
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).
ProgressReviews(form/rubric)¤tCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining
100%satisfactoryafteranynecessaryremediationplan(onepersonexitedtheprogramandanotherisabouttoenterremediation).
100%ofevaluationscoresindicatelevel2(mastery)
Thisprocessisconsistentlyidentifiedasastrengthforourprogram.Givenitsdevelopmentalnature,wecannotexpect100%fromeverystudent–particularlyintheirfirstyear.ButwedoexpectagradualimprovementbothinthequantitativeandqualitativeassessmentaswellasacontinuousgrowthpatternsintheevidenceoftheirworkintheCV.
ResultsDissemination(APTElement5)Animmenseamountoftime,energy,andresourcesarededicatedtoconductinghigh-qualityassessment.Fartoomanyresourcesarededicatedtoassessmentforresultstogounused.Thefirststepofusingresultsistoshareresultswithkeystakeholders,suchasfaculty,departmentheads,students,amongothers.Thus,programsshouldclearlyarticulateaplanforresultdissemination.CACREP,requiresthefollowing,under“EvaluationoftheProgram”:
Counseloreducationprogramfacultydisseminateanannualreportthatincludes,byprogramlevel,(1)asummaryoftheprogramevaluationresults,(2)subsequentprogrammodifications,and(3)anyothersubstantialprogramchanges.Thereportispublishedontheprogramwebsiteinaneasilyaccessiblelocation,andstudentscurrentlyintheprogram,programfaculty,institutionaladministrators,andpersonnelincooperatingagencies(e.g.,employers,sitesupervisors)arenotifiedthatthereportisavailable.
Therefore,themajorresultsfromtheAPTandoursystematicassessmentprocesswillbemadeaccessibleonourprogramwebsite,inadditiontootherdatawecollectforourCACREPreport,suchassiteevaluationinformation,exitsurveydata,andemploymentandgraduationrates.Ourreportwillalsobesharedwiththecounselingprogramfaculty,ourdepartmenthead,andmadeavailabletostudentsandothercounselingconstituentsviaourprogramwebsite.Finally,wedrawfromourpreviousyear’sAPTinordertocreateourCACREPAnnualReport,dueeachFall,sharedwithourstakeholders,postedonthewebsite(https://psyc.jmu.edu/counseling/supervision/index.html),anddocumentedforCACREP.TheinformationinthisreportwillbeusedtoinformtheFall2021postingonourwebsite.UseofResultsforLearning&DevelopmentalImprovement(APTElement6A)Ideally,programsuseassessmentresultstomakecurricularandpedagogicalchangesthattheybelievewillassiststudentsinbettermeetingthestudentlearningobjectives.Programsshouldspecifyaclear,detailedplanforusingresults.Inthisplan,programsmayconsiderdescribingtheplannedcurricularchangesbasedonresults,implementationstrategies,implementationdates,andwhy/howthecurricularchangesareexpectedtoimprovestudentlearning.InreflectionofthecurrenteraandCOVIDpandemic,theprogram’sleadership/assessmentcommitteehasbeenfocusingonimprovingthedeliveryofonlinetrainingtocounselinggraduatestudents.AccordingtothecounselingprogramsOnlineLearningAgreementthatwasdisseminatedtoallenrolledcounselingstudents:
Asweworktomaintaintheacademicrigorandintegrityofcounselortraininginadistanceenvironment,werecognizeeveryonewillneedtoadjustandadapttoanewwayoflearning.
PertheCACREP2016standards,counselingprogramshavebeenevaluatingandincorporatingthefollowingstandardsintoourcourseworkandclinicaltraining: Technology’simpactonthecounselingprofession(F.1.j.).
Modalitiesofclinicalsupervisionandtheuseoftechnology(B.2.g.).Ethicalandculturallyrelevantstrategiesforestablishingandmaintainingin-personandtechnology-assistedrelationships(F.5.d).
Overthepastyear,we’veexaminedwaystoimproveourassessmentofonlinelearningintheCounseling&Supervisionprogram.Thechangesthatwe’vemade/aremakinginclude:
1. Studenttelementalhealthcounselingtraining:allstudentsbeforepracticumandinternshipplacementsbeginmustparticipateandcompleteanonlinetrainingmoduleonthedeliveryoftelementalhealthservices.CompletionofthetrainingisuploadedtoourHUBmodule.Thisalignswithstandard(F.1.j.).
2. AdditionofSupervisionAssist:StudentsarenowabletousetheHIPAAplatformforclass,practicaandinternshipexperiences.Thisincludesrecordingsessionswithclientsandparticipationinindividualsupervision.ThisalignswithstandardsF.1.j.,B.2.g.andF.5.d.
• AssessmentofSupervisionAssistcanbeobtainedbycollectingfrequencydataofusers.
• WearestilllearningthenuancesoftheplatformandarestillrelyingoninformationcollectedviaCanvasandourProgressReviewsforassessment.Thereareafewkinksspecifictothediverseclinicalexperiencesofdoctoralstudents.
3. FacultyParticipatedinOnlineTrainingOpportunities:Priortothebeginningofthefall2020semester,allcounselingprogramfacultyagreedtocompleteindividualizedtrainingonteachingonline.
• Wecanuseteachingevaluationsandanecdotalinformationtohelpusassessthis.
4. AdditionofCANVASasalearningmanagementtoolinallcounselingclasses:Priortothebeginningofthefall2020semester,allcounselingprogramfacultydevelopedaCANVASpagefortheirassignedclasses
• UseofCANVAScanbeobtainedbycollectingfrequencydataofusers.
Animportantconsiderationwithour2019-2020APTalternativewastheoverallprogramchangesanticipatedinthePhDprogram.Thisisdifferentthanourothertwoprograms(SchoolCounselingandClinicalMentalHealthCounseling).ThePhDprogramismovingtowardalow-residencydeliverymodelwithcoursesbeingofferedinablendedsynchronous/asynchronousmodelaccompaniedbytwo“residency”weekendsbeginninginFall2021.ThisisdependentonapprovalofaSubstantiveChangeRequestwithCACREP.So,inadditiontotheitemsmentionedabovewhichareconsistentforallthreeprograms,wehaveamassivemigrationandcoursedevelopmentarchitecturaldrivethiscomingacademicyear.Thismayhavesomeimpactonassessment;however,itisdifficulttofullanticipateorarticulatethatfullyatthispoint.
Objective Changeincurriculum Anticipatedtimelineforimplementation Reason(s)forchange
KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.
Wearecurrentlycollectingdatafromconstituentstohelpinformourcurriculum,andininformalconsultationwithinternalandexternalstakeholders,weareaddingaseriesofworkshopsdesignedtoenhanceourstudents’preparationforclinicalpracticeincommunitysettings.OneofthecaveatstotheSubstantiveChangeRequestwithCACREPisthatwenotmakesubstantivechangestothecurriculum.Wearehavingtocalibratechangesindeliveryversus“normal”curricularchangesbasedondemonstratedfeedback.
Fall2021 We’rebasingthischangeonourobservationofemergingresearchinthefieldaswellasfeedbackfromgraduatesregardingrecentchangesandtrendsinthefield.FeedbackfromsitesupervisorsandrequirementsofCACREParethereasonweareengaginginadditionalstreamliningconversationswiththesites;although,wedoremaininconstantconversationwiththemthroughouttheacademicyear.
KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.
Nospecificchangebeyondmodifyingsyllabitoincludeemergingresearchaswellasadditionaltrainingindistanceortelehealthsupervisionpractices.
CurrentandFall2021. Workshopswillhelpalignthetrainingthatthemaster’sstudentsaredoingwiththesupervisioninsightneededbythedoctoralstudents.Theywillalsobeinclusiveoftelehealthsupervisionandtraining.Telehealthsupervisionandtrainingunitshavebeenupdatedinthesupervisionandsupervisioninternshipcoursestoreflectcurrentresearch.
KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.
Nospecificchangebeyondmodifyingsyllabitoincludeemergingresearch.Thispastacademicyearwechangedthewaydoctoralstudentsengageintheirfirstteachingexperience.Theynowpartnerwithafacultymemberforanentiresemester.Wehavefoundthisgreatlyassiststheminpreparationfortheirteachinginternship.Needtoexpandourpedagogyrelatedtodistancelearning.
CurrentandFall2021. Continueplacingnewdoctoralstudentswithafacultymembertoco-teachamaster’slevelcourseduringtheirfirstsemester.Wewillcontinuetomonitortheincreasedconfidence,competence,andpreparationfortheirteachinginternship.AdditionalmodulesandpracticeindistancelearningarebeingincorporatedintothePSYC853CounselorEducation(pedagogy)courses.
KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.
Nospecificchangeatthistimebutweareconsideringopportunitiesforadditionalresearchcollaboration,coursesectionsdedicatedspecificallytotheneedsourstudents.
CurrentandFall2021. Considerationofchangesisresource-basedandthereforechangesarenotbeingmadeatthistime.WedoneedtoworkmorecloselywithA&MfacultytomakesurewecontinuetobealignedwithCACREPrequirementsintheresearchcourses.TheDeptofGraduatePsychologyisalsoconductingasearchforaresearch-specificfacultymemberwhoshouldbeabletoassistwithraisingthebaronourdoctoralresearchexperiencesandcompetencies.
KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.
Moreopportunitiesarebecomingavailableinourfieldfor“emergingleaders”programsinavarietyofprofessionalorganizations.Wewant
Summer2021 We’rebasingthischangeonourobservationofemergingresearchinthefieldaswellasfeedbackfromgraduatesregarding
ourdoctoralstudentstobecomeinvolvedintheseinordertomakeadditionalprofessionalconnectionsanddocumentableexperienceasleadersandadvocates.
recentchangesandtrendsinthefield.Weareaddingopportunitiesforleadershipandadvocacyexperiencestocounttowardinternshiphours.
KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.
Nospecificchangeatthistime;however,weareworkingtostrengthenthisareaandaregivingconsiderationtoadditionaloptionsforelectives,service,andtraining.
Spring2022 Spring2022willbringanewinstructortotheCrisis&Emergencycourseandwillincludeclimate-informeddisasterresponse,theneurobiologyoftraumaandACEs,systemicandculturaltrauma,aswellasincorporatingcrisisintosupervisionandteaching.
KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).
Nospecificchangeatthistime.Thishastypicallybeenanareathatouraccreditorshavepraisedasbeinghighlyeffectivesowearenotinclinedtomakeanyadjustmentsatthistime.
InSpring2019,webeganuploadingthesetoaCanvasassessmentsitesobothfacultyandstudentscanhaveaccesstoallpriorProgressReviews.Thisallowsustorefertothemasneededandcompareresultsacrosstime.Wecontinuetoexploreoptionsforusinganothersurveymethodtobettercollect,analyze,andpreserveprogressreviewdata.
UseofResultsforAssessmentImprovement(APTElement6B)Assessmentprocessesmustcontinuallyevolvetoaccommodatenewresearch,changingstudentdemographics,evolvingfaculty/departments,etc.Thus,changeswilllikelybemadetotheassessmentprocessfromyeartoyear.Changesmayincludemodifyingobjectives,changingmeasures,changingwhichstudentsareassessed,changingdatacollectionprocedures,etc.Programsshouldspecifypastchangesaswellasplansforfuturechanges,andmayconsiderprovidingaplanforimplementationoffuturechanges.
Change Anticipatedtimelineforimplementation Reason(s)forchange
WerecentlybegantheprocessofclarifyingourassessmentplaninresponsetoourDecember2018self-studysubmission.Thisyearwehavebeenimplementingtheself-studyandproblem-solvinganyissuesthatemerge.Themajorchangesincluded1)addingprogramspecificKeyCounselorEducationandSupervisionCompetencies(KPIs)toourexistingassessmentplan;2)checkingtheaccuracyofourcurriculummapofobjectiveswithcorrespondingcoursesandexperiences;3)ensuringthateachobjectiveisassessedattheintroductory,practice,andmasterylevelsforallstudents;4)determiningthemostefficientandeffectivemethodforgatheringandmaintainingthesedata;and5)revisingourcommunicationwithexternalstakeholders(alumnae;currentemployers;localpractitioners)toensureongoingapplicabilityofourtrainingtotheneedsofcounselingprogramsacrossthecountry.
OngoingthroughSpring2018
PleasenotethattheassessmentplanwasfinalizedaspartofoursubmissionofourCACREPSelf-StudyinDecember2018.ForFall2018andSpring2019,wecollectedthedataAlthoughwearecurrentlyusingsomeofthesameprocedureswehaveusedinthepasttogatherdataregardingourstudents’progress(comprehensiveexams,practicumandinternshipevaluations,progressreviews),wehavefelttheneedtocreateamoreefficientsystemthatwillallowforeasyaggregationanddisaggregationofdatatobetterenableustoadvisestudentsandnotetrendsacrosstime.Atthispoint,wehavecreatedassessmentsitesonCanvastocollectandshareallKPI-relateddataamongfacultyandwithstudents.AscreenshotofthatpageaswellasascreenshotofoneoftheKPIassignmentrubricsareattachedtothisreport.WearefindingthistobeausefulwaytoassessstudentsusingtheKPIrubricsandallowfortheopportunitytoalsouploadKPIassignmentforreference.Thedataanalysisfunctionwillallowustolookatdataovertimeandacrosscohorts.Wehavealsopurchased(throughstudentfees)aclinicalsupervisionprogramcalledSupervisionAssist.Thisallowsustohouseallclinicalworkforallstudentsincludingclientcounselingvideos,supervisionsessions,connectionstositesupervisors,allcontractsandevaluations.Thisremainsaworkinprogressasweacclimatedtothefullcapacityofthisplatform.Wewelcomefeedbackasweworktostrengthenouroverallassessmentplan.