phd in counseling & supervision annual report 2018-2019

36
CACREP ANNUAL REPORT Academic Degree Program: PhD in Counseling & Supervision Department Head: Robin Anderson Assessment Coordinator: Debbie C. Sturm

Upload: others

Post on 03-Dec-2021

2 views

Category:

Documents


0 download

TRANSCRIPT

CACREPANNUALREPORT

AcademicDegreeProgram:PhDinCounseling&Supervision

DepartmentHead:RobinAnderson

AssessmentCoordinator:DebbieC.Sturm

ThePh.D.inCounselingandSupervisionisaprogramcommittedtotrainingcounseloreducators,supervisors,andadvancedpractitionerswithexpertiseinthetheory,researchandpracticeofcounseling.Ourmissionistocreatedynamicandinnovativeleaderswhowillserveascatalystsforprogressinaddressingthetremendouscounselingneedsofindividuals,families,institutions,andcommunities.Thestudents,facultyandstaffmembersofthisprogramvaryinage,class,gender,ethnicity,race,religion,sexualorientation,andphysicalabilities.Puttingourprinciplesintopractice,thesediverseindividualscreatealearningcommunityinwhichstudentsthrivepersonally,growprofessionallyandshareacommoncommitmenttocounseling.Wechallengeourstudentstocontinuetheirlife-longjourneysofexploringpossibilities,refiningskillsandmaintainingtheirprofessionalvitalitythroughouttheircareersascounselors.Weencouragethemtosupportoneanotherintheformidabletaskofbecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining.

ThePh.D.inCounselingandSupervisionwasapprovedbytheStateCouncilofHigherEducationinVirginia(SCHEV)andimplementedin2011withacohortoffivedegree-seekingstudents.Thecurriculumfocusesoncounseloreducation,supervisionstrategies,leadershipskills,counselingservices,andresearchmethods.Eligibleapplicantsarepracticingcounselorswhohavecompletedanadvanceddegree(M.A.,M.S.orEd.S.)fromacounselingmaster’sdegreeprogramand60credithoursofgraduatecounselingcourseworknecessaryforlicensureasaprofessionalcounselor,withpreferencegiventograduatesofCACREPaccreditedprograms.Theemphasisoncrisiscounselingandemergencyservicesisalsoauniquefeatureoftheprogram.

Duringthe2018-2018academicyear,16studentshavebeenenrolledinthePh.D.inCounselingandSupervisionProgram.Sincetheimplementationoftheprogram,eightdoctorateshavebeenawarded.Duringthepasttwoyears,weadmittedatotalof7students.Ourfull-timestudentsareexpectedtocompletetheprograminthreeyearsandourpart-timestudentsplantocompletetheprograminfiveyears.Duringthefirstfouryearsoftheprogram,ourcombinedcompletionrateforfull-timeandpart-timestudentsinthePh.D.inCounselingandSupervisionProgramhasbeen89%.Allofouralumnireportthattheyarelicensedandworkingeitherfull-timeorpart-timeincounseloreducation,supervision,andcounselingpractice.

ThePhDinCounseling&Supervisionincludesa48-hourcurriculumbasedonthe2016standardsestablishedbytheCouncilforAccreditationofCounselingandRelatedEducationalPrograms(CACREP),whichrequiresdemonstrationofmasteryacrossfivecorecounselingareasaswellasatleastoneadditionalareaspecificallyrelatedtothespecificmissionorvaluesoftheprogram.Inourcase,theadditionalareacurrentlyindevelopmentisCrisis,Emergency&Disasterresponse.Counselingprogramsarealsoaskedtoincludeprogram-specificstandardsthatareconsonantwiththecultureandexpectationsofthatspecificprogramaswellasaprofessionaldispositionsmetricintheformofregularprogressreviews.Forthesakeofclarityandconsistency,theprogramspecificexpectationshavebeenmapped,whereappropriatetotheCACREPstandards(seeAPTElement1A,below).Inadditiontootherconsiderations,CACREPrequiresevaluationofKeyPerformanceIndicators(KPIs)ofstudentlearningineachoftheabovementionedareasviamultiplemeasuresandovermultiplepointsoftime.Thespreadsheetdocumentingourcurrentplanforthisevaluationisattached,withspecificinformationrelevanttothisAPTincludedbelow.ThisassessmentplanwasfirstimplementedbeginninginFall2018intandemwithourprocessofsubmittingtheSelf-StudytoCACREP.Wearecontinuallycollectingdatafromconstituentstohelpinformourcurriculum,andininformalconsultationwithinternalandexternalstakeholders,weareaddingaseriesofworkshopsdesignedtoenhanceourstudents’preparationforemploymentasCounselorEducatorsoradvancedsupervisorsinclinicalsetting.

TheAssessmentPlanisintendedtoprovideaframeworkofprogramevaluationandassessmentthatalignswithJamesMadisonUniversity’s(JMU)counselingprograms’missionandidentifiedstudentlearningobjectives.ThecounseloreducationfacultyatJMUarecommittedtoprogramqualityandcontinuousimprovement.Thus,theprocessofevaluationandassessmentisintegratedinallcomponentsofgraduatetrainingandprogramplanning.Thisincludesevaluationprocessesforadmissionsdecisions,studentmatriculation,courseeffectiveness,facultyperformance,supervisorperformance,andemployerandalumniexperiences.PertheAssessmentPlan,assessmentandevaluationwithintheJMUcounselingprogramsarealignedwiththeCouncilfortheAccreditationofCounselingandRelatedEducationalPrograms(CACREP)standards.JMUcounselingfacultymeettwiceamonthtodiscussCACREPstandards,reflectoninfusingthesestandardsintoprogramandstudentevaluationassessmentprocesses,andcollectingthisdataacrossprograms.Allcollecteddataareanalyzedbyprogramfacultyandreportedtostakeholders(e.g.,administration,accreditationteams,alumni,employers,potentialstudents)asrelevant.StakeholdersprovideprogramandcurricularfeedbacktocontinuallyimprovethecounselingtrainingprogramsatJMU.Theprocessofevaluationandassessmentisdescribedinfurtherdetailbelow.EvaluationoftheCounselingProgramsApproach

TheAssessmentPlanwasdesignedbyfacultyasaformalevaluationprocessthatdocumentstheirempiricallybasedapproachforsystematicallyevaluatingprogramobjectives,keyperformanceindicators(KPIs),studentlearningwithanemphasisonacademicperformanceandprofessionaldisposition,andfaculty/supervisorperformance.TheAssessmentPlanincludesadescriptionofthedatacollected,themethodforhowthedatawascollected,adescriptiononhowthedatawasanalyzed,howthedatainformedcurriculum,,andprogramimprovementsacrossallthreecounselingprograms.Thevisualchartbelowdescribesthefourcomponentsofthisevaluationplan(A.1.ProgramEvaluation;A.2.StudentAssessment;A.3.EvaluationofFaculty;andA.4.EvaluationofSupervisors)andprovidesinformationonthetypesofassessmentsutilizedtocollectevaluativedata.

StudentLearningObjectives(APTElement1A&1B)Studentlearningobjectives(SLOs)arestatementsindicatingwhatstudentsshouldknow,think,ordoasaresultofparticipatinginanacademicdegreeprogram.SLOsshouldbestudent-centeredandbewrittenclearlywithprecise,measureableverbs.ThereisnosetnumberofSLOsrequiredfortheAPT.

StudentLearningObjectiveAsaresultofparticipatingintheDoctoralPrograminCounseling&Supervision,studentsgraduatingwithanPhDinCounseling&Supervisionwill:

KPI1Counseling:Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.Takeresponsibilityforembodyingandpracticingprofessionalandethicalstandardsatthedegreeofmasterybyeffectivelydemonstratingcounselingskillsincludingtheabilitytoapplycounselingtheoriesandevidence-basedmethods,evaluateclientprogress,anddemonstrateethicalandculturally

competentcounselingpractice.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.

AlignedwithCACREPCounselorEducationandSupervisionStandards:B.1.a-f1. COUNSELING

1. scholarlyexaminationoftheoriesrelevanttocounseling2. integrationoftheoriesrelevanttocounseling3. conceptualizationofclientsfrommultipletheoreticalperspectives4. evidence-basedcounselingpractices5. methodsforevaluatingcounselingeffectiveness6. ethicalandculturallyrelevantcounselinginmultiplesettings

KPI2Supervision:Knowledgeandskillsforeffectiveclinicalsupervision.Takeresponsibilityfordevelopingapersonal,ethical,research-basedandculturallyrelevantidentityasasupervisorbydemonstratingknowledgeoftheoriesandmodelsofclinicalsupervisionandtheskillsnecessarytoprovideeffectiveclinicalsupervision.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.

AlignedwithCACREPCounselorEducationandSupervisionStandards:B.2.a-k

2. SUPERVISION1. purposesofclinicalsupervision2. theoreticalframeworksandmodelsofclinicalsupervision3. rolesandrelationshipsrelatedtoclinicalsupervision4. skillsofclinicalsupervision5. opportunitiesfordevelopingapersonalstyleofclinicalsupervision6. assessmentofsupervisees’developmentallevelandotherrelevantcharacteristics7. modalitiesofclinicalsupervisionandtheuseoftechnology8. administrativeproceduresandresponsibilitiesrelatedtoclinicalsupervision9. evaluation,remediation,andgatekeepinginclinicalsupervision10. legalandethicalissuesandresponsibilitiesinclinicalsupervision11. culturallyrelevantstrategiesforconductingclinicalsupervision

KPI3Teaching:Knowledgeandskillsrelevanttoeffectiveteaching.Takeresponsibilityfortrainingascounseloreducatorswithexpertiseinthetheory,researchandpracticeofcounselingasevidencedbydemonstratingaknowledgeofpedagogyandskillsrelevanttoteachingincludinginstructionaldevelopment,implementation,andevaluationmethodsincounseloreducation.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.

AlignedwithCACREPCounselorEducationandSupervisionStandards:B.3.a-i.

3. TEACHING1. rolesandresponsibilitiesrelatedtoeducatingcounselors2. pedagogyandteachingmethodsrelevanttocounseloreducation3. modelsofadultdevelopmentandlearning4. instructionalandcurriculumdesign,delivery,andevaluationmethodsrelevant

tocounseloreducation

5. effectiveapproachesforonlineinstruction6. screening,remediation,andgatekeepingfunctionsrelevanttoteaching7. assessmentoflearning8. ethicalandculturallyrelevantstrategiesusedincounselorpreparation9. theroleofmentoringincounseloreducation

KPI4Research&Scholarship:Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.Weencouragestudentstosupportoneanotherprofessionallyinthetaskofbecomingleaderswhoadvancethecounselingprofessionthroughresearchandinnovationasevidencedbydemonstratingtheknowledgeandskillsnecessarytoengageinresearchandscholarship,includingquantitativeandqualitativeresearch,thatadvancedthefieldofcounseling,counseloreducation,andsupervision.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.AlignedwithCACREPCounselorEducationandSupervisionStandards:B.4.a-l

4. RESEARCHANDSCHOLARSHIP1. researchdesignsappropriatetoquantitativeandqualitativeresearchquestions2. univariateandmultivariateresearchdesignsanddataanalysismethods3. qualitativedesignsandapproachestoqualitativedataanalysis4. emergentresearchpracticesandprocesses5. modelsandmethodsofinstrumentdesign6. modelsandmethodsofprogramevaluation7. researchquestionsappropriateforprofessionalresearchandpublication8. professionalwritingforjournalandnewsletterpublication9. professionalconferenceproposalpreparation10. designandevaluationofresearchproposalsforahumansubjects/institutionalreview

boardreview11. grantproposalsandothersourcesoffunding12. ethicalandculturallyrelevantstrategiesforconductingresearch

KPI5Leadership&Advocacy:Knowledgeandskillsforeffectiveleadershipandadvocacy.Weencouragestudentstosupportoneanotherasprofessionalsinthetaskofbecomingleaderswhoadvancethecounselingprofessionthroughservice,innovation,andadvocacyasevidencedbydemonstrationofsufficientknowledgeandskillstobeeffectiveinadvocacyandleadershipinthecounselingprofession.Evidenceofmasteryrequiresascoreofatleast2onthemeasuresidentifiedintheaccompanyinggrid.AlignedwithCACREPCounselorEducationandSupervisionStandards:B.5.a-l.

5.LEADERSHIPANDADVOCACY1. theoriesandskillsofleadership2. leadershipandleadershipdevelopmentinprofessionalorganizations3. leadershipincounseloreducationprograms4. knowledgeofaccreditationstandardsandprocesses5. leadership,management,andadministrationincounselingorganizationsandother

institutions

6. leadershiprolesandstrategiesforrespondingtocrisesanddisasters7. strategiesofleadershipinconsultation8. currenttopicalandpoliticalissuesincounselingandhowthoseissuesaffectthedaily

workofcounselorsandthecounselingprofession9. roleofcounselorsandcounseloreducatorsadvocatingonbehalfoftheprofessionand

professionalidentity10. modelsandcompetenciesforadvocatingforclientsattheindividual,system,andpolicy

levels11. strategiesofleadershipinrelationtocurrentmulticulturalandsocialjusticeissues12. ethicalandculturallyrelevantleadershipandadvocacypractices

KPI6Crisis&DisasterResponse:Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.Weencouragestudentstoembracetrainingasadvancedpractitionerswithexpertiseinthetheory,researchandpracticeofcounselingasitrelatestocrisis,emergencyanddisasterresponseasevidencedbydemonstrationofsufficientknowledgeandskillsoftheimpactofcrisisanddisasteronclientsaswellasknowledgeandskillinrespondingtocrisisanddisastersclinicallyandsystemically.(NOTE:Thisisanareawearestilldeveloping.AtthispointwehaveoneclassandareworkingtoaddtothatwithelectivesandKPIsinothercourses.)

AlignedwithCACREPCounselorEducationandSupervisionStandards:

B.5.f.leadershiprolesandstrategiesforrespondingtocrisesanddisastersB.5.g.strategiesofleadershipinconsultationB.5.h.currenttopicalandpoliticalissuesincounselingandhowthoseissuesaffectthedailyworkofcounselorsandthecounselingprofession

KPI7KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.

AlignedwithCACREPCounselorEducationandSupervisionStandards4.G.

Thecounseloreducationprogramfacultysystematicallyassesseseachstudent’sprofessionaldispositionsthroughouttheprogram.Theassessmentprocessincludesthefollowing:(1)identificationofkeyprofessionaldispositions,(2)measurementofstudentprofessionaldispositionsovermultiplepointsintime,and(3)revieworanalysisofdata.

Course/learningExperiences(APTElement2)IfstudentsareexpectedtomeetthestatedSLOs,theymustbeprovidedwithlearningopportunitiestoassisttheminmeetingtheSLO.LearningopportunitiesshouldbeclearlylinkedtoSLOs,andallSLOsshouldbecoveredbyatleastonelearningopportunity.Thisisoftenreferredtoasprogramtheory.Twooptionsforacurriculummapareprovidedbelow.Youmaychooseeitheroptionforpresentingyourcurriculummap.

Studentlearningobjective Courses/Experiencesmappedtotheobjective

Introductory Practice MasteryKPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper

PSYC882Counselingpracticumperformanceassessment

PSYC892CounselingInternshipperformanceassessmentComprehensiveExam:CounselingEssayandCasePresentation

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignment

PSYC892SupervisioninternshipSuperviseeConceptualization

ComprehensiveExam:SupervisionEssayandCasePresentation

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTraining

PSYC892SyllabusandTeachingPhilosophy

PSYC892TeachingPerformanceEvaluationComprehensiveExam:TeachingEssayandCasePresentation

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC605FinalExamPSYC840ResearchDesignProposals

PSYC881LiteratureReviewPSYC881ResearchAgenda

ProposalDissertationDissertation

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignment

PSYC810Assignment:EducationSession

ComprehensiveExam:Leadership&AdvocacyEssayandDemontrationofWork

KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.

PSYC866CrisisPresentation TBD(Thiselementisstillbeingdeveloped).

TBD(Thiselementisstillbeingdeveloped).

KPI7.KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthical

AssessedattheconclusionofeachsemesterviatheProgressReviewFormandFacultyReviewProcess.AlsoCVtoevidencetheir

AssessedattheconclusionofeachsemesterviatheProgressReviewFormandFaculty

AssessedattheconclusionofeachsemesterviatheProgressReviewForm

BehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.

professionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining

ReviewProcess.AlsoCVtoevidencetheirprofessionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining

andFacultyReviewProcess.AlsoCVtoevidencetheirprofessionaldevelopmentascounseloreducatorsthroughservice,research,innovation,advocacy,andtraining

*Note:forthistable,programsmayplacean“X”intotheappropriateintersectingboxestoindicatewhichcourse/learningexperiencesmaptowhichobjective(s).Foreithertable,programsmayspecifycoverageoftheobjectivebycourse/learningexperiencesbyplacinga“1”,“2”,or“3,”etc.,intotheappropriateboxes,wherea1indicatesminimalcoverageandhigherscoresindicatemorecoverage.AssessmentMeasures(APTElement3A&3B)ToobtainresultsthatareusefulforevaluatingwhetherstudentsmetthestatedSLOs,instrumentsmustbeselectedtoelicitthedesiredknowledge,skills,orattitudesfromstudents.AllSLOsshouldbemeasuredbyatleastoneinstrument.Moreover,toobtainthestrongestevidenceofstudentlearning,SLOsshouldbemeasuredbyadirectmeasureofstudentlearning.

Objective DescriptionofInstrumentusedtoassessobjective

Direct/Indirect

1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(RubriconCanvas)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam

I-assessmentisbasedonwrittenproductD–studentisobservedD–studentisobservedI-assessmentisbasedonwrittenproduct

2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualizationComprehensiveExam

AllthreeinvolvebothD&I:D–studentisobservedI-assessmentisbasedonwrittenproduct

3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTraining(RubriconCanvas)PSYC892SyllabusandTeachingPhilosophyPSYC892TeachingPerformanceEvaluationComprehensiveExam

Allfourinvolveawrittenproduct(I)andsomedegreeofobservation(D):I-assessmentisbasedonwrittenproductD–studentisobserved

4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC840ResearchDesignProposalsPSYC609ResearchProposalPSYC605FinalExamPSYC881LiteratureReviewPSYC881ResearchAgendaProposalDissertationDissertation

I-assessmentsarebasedonwrittenproductI-assessmentisbasedonwrittenproductI-assessmentisbasedonwrittenproductANDD–studentisobserved

5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignment(RubriconCanvas)PSYC810Assignment:EducationSession(RubricforwrittenandobservedelementsonCanvas)ComprehensiveExam

I-assessmentisbasedonwrittenproductI-assessmentisbasedonwrittenproductD–studentisobservedD–studentisobservedI-assessmentisbasedonwrittenproduct

6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.

PSYC866CrisisPresentation I-assessmentisbasedonwrittenproductANDD–studentisobserved

7.KeyProfessionalCompetenciesandDispositions:Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.

ProgressReviews&CVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining

I-assessmentisbasedonwrittenproductANDD–studentisobserved

DesiredResults(APTElement3C)Toaidininterpretationofresults,itishelpfultodeterminearesultthatprogramshopestudentswillmeet(i.e.80%ofstudentswillpassanexam,orstudentswillincreasebyatleast10pointsfromthebeginningoftheprogramtotheendoftheprogram).Ideally,programswillhaveadesiredresultforeachobjectiveandwilljustifytheirdesiredresultbasedonexternalresearch,facultyconsensus,previousyears’results,etc.

Objective Instrument Desiredresult Justificationfordesiredresult

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam

90%orgreaterratingsof2inordertobeconsideredtohavepassedthiselement.100%passresultinordertopromotetointernship100%passresultinordertosuccessfullycompleteinternshipandbeeligibleforcomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualizationComprehensiveExam

90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetosupervisioninternship90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTrainingPSYC892SyllabusandTeachingPhilosophyPSYC892TeachingPerformanceEvaluationComprehensiveExam

90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetosupervisioninternship90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Recentresearch(Goodwin,Coyne,&Constantino,2018)hashighlightedthecontextualdemandsforcompetence;thereforestudentsareprovidedconsultationandregularlayersofsiteandfacultysupervisionaswellasevaluationtoensuretheymeetthisstandard

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC840ResearchDesignProposalsPSYC609ResearchProposalPSYC605FinalExamPSYC881LiteratureReviewPSYC881ResearchAgendaProposalDissertation

90%ofevaluationscoresindicatelevel2(mastery)90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullyproposedfordissertationandsuccessfullydefendeddissertation.By“2ndattempt”,werefertosituationsinwhichstudentshavesignificantfeedbacktoaddressbeforeconsideredpassing.

ThespecificconsiderationscoveredintheresearchcoursesseriesaredesignedtocorrelatewithCACREPresearchknowledgeandskillsandaredesigned/measuredbasedontheexpertiseoftheA&Mfacultywhoteachthosecourses.Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignmentPSYC810Assignment:EducationSessionComprehensiveExam

90%ofevaluationscoresatlevel2,indicatingsufficientmasteryandmovetocomprehensiveexams100%passon2ndattemptinordertobeconsideredtohavesuccessfullypassedcomprehensiveexams.

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.

KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.

PSYC866CrisisPresentation 90%ofallevaluationsatlevel2,indicatingmastery

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimately

tosuggestthestudentispreparedforpost-graduationpractice.

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).

ProgressReviews(form/rubric)&currentCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining

100%satisfactoryafteranynecessaryremediationplan

Combinedfacultyexperiencehasindicatedthesepassratesarenecessaryforsuccessfulprogressthroughtheprogramandultimatelytosuggestthestudentispreparedforpost-graduationpractice.Progressreviewsincludefacultyevaluationsofstudents’ProfessionalandEthicalBehaviorandAcademicBehavior.Thespecificdomainsareinformedbycounselingdispositionsidentifiedbyouraccreditingbodyandcurrentresearchthatdefineappropriateattitudesandcapabilitiesthatstudentsbringtotheircounselortraining.

DataCollection(APTElement3D)Sounddatacollectionproceduresareintegralforobtaininghighqualityresults.Datacollectionconsiderationsincludewhichstudentsweresampled,howmanystudentsweresampled,whetherthestudentswererepresentativeofthestudentstowhichinferenceswillbemade,whetherdatawerecollectedatonetimepointormultipletimepoints,whetherstudentsweremotivatedtogivebesteffortontheassessments,etc.Datacollectionproceduresmaydifferbasedonwhetherselected-responseorperformanceassessmentsareadministered.Forexample,ifaperformanceassessmentisadministered,itmayalsobeusefultoincludemultipleratersandratertrainingpriortoscoringthestudentartifacts.

Objective Instrument Studentssampled Samplesize Timepoint(s) Motivation

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(Rubric)

Allstudents

2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)

PSYC860:1styear/1stsemester

Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,

PSYC882Counselingpracticumperformanceassessment

PSYC882:1styear/2ndsemester

PSYC892CounselingInternshipperformanceassessment

PSYC892:2ndyear/1stsemester

andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.

ComprehensiveExam(CE)Rubric

CE:Aftercompleting2ndyear

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignment

Allstudents

2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)

PSYC853:1styear/2ndsemester

Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.

PSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)

PSYC892:2ndyear/1stsemester

ComprehensiveExam(CE)Rubric

CE:Aftercompleting2ndyear

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTrainingRubric

Allstudents

2(1styearcohort)2(2ndyearcohort)3(3rdyearcohort)

PSYC863:1styear/1stsemester

Studentswillnotprogressintheprograminsequenceswithoutcompletingeachofthesestepssuccessfully,andtheywillnotgraduatewithoutdemonstratingsufficientmasteryininternship.Thustheyarehighlymotivated.

PSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessment

PSYC892:2ndyear/1stsemesterand/or2ndsemester

ComprehensiveExam(CE)Rubric

CE:Aftercompleting2ndyear

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC605FinalExam

Allstudents

2(1styearcohort)2(2ndyearcohort)2from2ndyearcohortforproposalsand3from3rdyearcohortfordissertations

PSYC605:1styear/1stsemester

Thisinstrumentincludeoneevaluationfortheidentifiedthatcontributestotheoverallevaluationofarequiredcourse.StudentsareusuallyfairlymotivatedtoearnanAinthesecourses.Thisistruefor840,609,605,and881.WehaverubricsforDissertationProposalandDissertation.Allelementsmustbesuccessfullypassedinordertobeconsideredcompleteandstudentsarehighlymotivatedtobesuccessfulinthisprocess.

PSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)

PSYC840/609:2ndyear/1stsemesterand/or2ndsemesterPSYC881:2ndyear/2ndsemester

ProposalandDissertation(Rubrics)

Aftercompleting2ndyear

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubric

Allstudents

2(1styearcohort)2(2ndyearcohort)3from2ndyearcohortforproposals

PSYC8621stsummerPSYC8102ndsummer

Thisinstrumentsincludeoneevaluationfortheidentifiedthatcontributestotheoverallevaluationofa

ComprehensiveExamRubric

Aftercompleting2ndyear

requiredcourse.StudentsareusuallyfairlymotivatedtoearnanAinthesecourses.Studentswillnotprogressintheprograminsequenceswithoutcompletingcomprehensiveexamssuccessfully.Thustheyarehighlymotivated.

KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.

PSYC866CrisisPresentation

Allstudents

2(1styearcohort)

PSYC866hasbeeninthe1stsummerbutisbeingmovedtothe1stfalleffectiveFall2018.

Thisinstrumentisoneindicatorconsideredinthecourseevaluation.TheyaremotivatedtoearnanAinthecourse.

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).

ProgressReviews(form/rubric)&currentCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining

Allstudents

2(1styearcohort)2(2ndyearcohort)3from3rdyearcohort1from4th+year

Eachsemester Progressreviewsarecompletedbyfacultyeachsemesterandaddresscounselingdispositions.Ideally,studentsareintrinsicallymotivatedthroughtheirinterestinthefieldtoexhibittheexpecteddispositionsacrossacademicandexperientialdomains.

ValidityEvidence(APTElement3E)Validityevidencecreatesastrongercaseforeventualinferencestobemadefromscores.Withoutvalidityevidence,itisdifficulttodeterminewhetherassessmentsaretrulymeasuringtheintendedknowledge,skills,andabilitiesprogramshavedeemedimportantforstudents.Importantvalidityevidenceincludes,butisnotlimitedto,reliabilityestimates(e.g.Cronbach’salphaorinterraterreliability),correlationswithothervariables(e.g.,professionalcertificationexamresultsorcoursegrades),andfaculty/expertevaluationofassessmentcontent.Theprogramfacultymemberstakeahermeneuticapproachtocomprehensiveexamandotherstudentevaluationratingsandthereforegroupconsensusisrequiredpriortothereportingoffinalratings.Forthisreasonwedonotcalculateinter-raterreliability.Facevalidityisassumedwithsite-supervisoranduniversity-supervisorevaluationsofstudentsinpracticumandinternship.ThisisduetothefactthatevaluationitemsaredrawnfromtheCACREPDoctoralCounselorEducationandSupervisionstandards(beginningFall2016),whicharethestandardsbywhichourprogramisevaluatedandaccredited.Further,theCACREPstandardsarethebasisforthedesignofour

coursesandourstudents’evaluationsacrossthecurriculum.Constructvalidityappliestothestandards,asexpertsinthefielddeterminetheCACREPaccreditationstandards.Constructvalidityisalsomaintainedasfacultyexpertiseresultsindeepknowledgeofthefieldanditsassociatedstandards.Withsomeexceptions,notedbelow,thefacultywhoteachcoursesassociatedwiththeobjectivesidentifiedherearecorefaculty,whoarelicensedmentalhealthproviderswithdoctoralleveltraininginourfieldofexpertise.Corefacultymembersteachcoursesthatarealignedwiththeirprofessionalexperienceandexpertiseandthereforehavesufficienttrainingandcompetencetoassesslevelsofstudentmastery.Incasesinwhichcorefacultymembersarenotavailabletoteachthesecourses,weemployadjunctfacultymemberswhoarespecificallychosenfortheirtrainingandexperiencerelatedtothecoursecontent.Further,allcorefacultymembersgatherattheendofeachsemestertoevaluateeverystudent’sprofessionaldispositionsandourkeyCounselorEducationandSupervisioncompetenciesandprofessionaldispositionsviaourProgressReviewprocess.Ourcombinedevaluationsassistusindeterminingthevalidityofthespecificmeasuresidentifiedinthetablebelow.

Objective Instrument InstrumentEvaluator Progress

reviewrelevance?

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric

PSYC860AssignmentRubricinCanvasPSYC882PracticumPerformanceAssessmentPSYC892InternshipPerformanceAssessment(Counseling)ComprehensiveExamRubric(CounselingCompetenciesSectionI)

PSYC860CoreFacultyMemberPSYC882CoreFacultyMemberPSYC892CoreFacultyMemberCECommitteeof2CoreFacultyMembers

Yes

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric

PSYC853AssignmentRubricinCanvasPSYC892InternshipPerformanceAssessment(Counseling)ComprehensiveExamRubric(SupervisionCompetenciesSectionII)

PSYC853CoreFacultyMemberPSYC892CoreFacultyMemberCECommitteeof2CoreFacultyMembers

Yes

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric

PSYC863AssignmentRubricinCanvasPSYC892AssignmentRubricinCanvasPSYC892InternshipPerformanceAssessment(Teaching)ComprehensiveExamRubric(SupervisionCompetenciesSectionIII)

PSYC863CorefacultymemberPSYC892CoreFacultyMember&SiteSupervisorCECommitteeof2CoreFacultyMembers

Yes.

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)

PSYC605AssignmentRubricinCanvasPSYC840AssignmentRubricinCanvasPSYC609AssignmentRubricinCanvasPSYC881AssignmentRubricsinCanvasDissertationProposalRubric(partofComprehensiveExamRubric)DissertationDefenseRubric

PSYC605CoreA&MFacultyPSYC840&609DeptofGraduatePsychologyFacultyPSYC881CoreFamilyMembersCommitteeof2CoreFacultyMembers

Yes

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric

PSYC862AssignmentRubricinCanvasPSYC810AssignmentRubricinCanvasComprehensiveExamRubric(Section5forLeadership&Advocacy)

PSYC862&810CoreFacultyMembers2CoreFacultyMembers

Yes

KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.

PSYC866CrisisPresentation PSYC866AssignmentRubricinCanvas

CoreFacultyMembers

Yes

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).

ProgressReviews(form/rubric)&currentCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining

ProgressReviewDocument(Rubric)

Studentself-evaluation;ReviewbyallCoreFacultyMembers;FeedbackfromAdvisor

Yes

Results(APTElement4A&4B)Resultsareusedtoconveytostakeholdershowwellstudentsmetthestatedstudentlearningobjectives.Thus,resultsshouldbeclearlypresentedinrelationtothestudentlearningobjectives.Often,historicalresultsprovidecontextforthecurrentyear’sresultsandprovideinsightaboutstudentlearningtrends.

HistoricalResults

Objective Instrument Priorresults Priorresults Thisyear’sresults

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric

50%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

50%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

50%passatPracticelevel75%passatPracticelevel100%passatMasterylevel

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevelon1stattempt100%passatMasterylevel

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(100%on1stattempt)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)

KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.

PSYC866CrisisPresentation 100%ofallevaluationsatlevel2practicelevel

100%ofallevaluationsatlevel2practicelevel

100%ofallevaluationsatlevel2practicelevel

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounseling

ProgressReviews(form/rubric)&currentCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,

80%satisfactoryafteranynecessaryremediationplan

80%satisfactoryafteranynecessaryremediationplan

80%satisfactoryafteranynecessaryremediationplan(onepersonenteredremediationand

ProgramsStudentProgressReviews).

innovation,advocacy,andtraining

istakingaleaveofabsense).

Results(APTElement4A&4B)Inadditiontopresentingresults,itisimportanttointerpretwhattheresultsmean,ofteninthecontextofstudentlearningobjectives.Interpretationsshouldmakereferencetothespecifieddesiredresults,aswellascurricular/pedagogicalchanges.

Interpretations

Objective Instrument ActualResult DesiredResult

Interpretation

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

PSYC860AdvancedTheoreticalOrientationPaper(Rubric)PSYC882CounselingpracticumperformanceassessmentPSYC892CounselingInternshipperformanceassessmentComprehensiveExam(CE)Rubric

100%passatPracticelevel100%passatPracticelevel100%passatPracticelevel100%passatMasterylevel

100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt

Wefeelconfidentinourabilitytohelpstudentslearnthecorecompetenciesassociatedwithknowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.Wealsorelyonthecombinationofdataobtainedfromboththecomprehensiveexamandtheinternshipexperiencestomakeourdeterminationsregardingstudentlearninginthisarea.Afterall,theperformanceincounselinginternshiphasadirectrelationshiptothelearningthatoccurswithinthisKPI.

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

PSYC853CaseConceptualization/LiteratureReviewAssignmentPSYC892SupervisioninternshipSuperviseeConceptualization(Rubric)ComprehensiveExam(CE)Rubric

59%passatPracticelevel90%passatPracticelevel100%passatMasterylevel

50%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt

Studentsthisyearpresentedacuriosityandmotivationregardingthissubjectmatterthatenabledthemtoprogressfurtherthansomepreviousclasses.GiventheinterconnectednatureofthePhDprogramlearning,astudentwhostruggledincounselingpracticumwillalsolikelyexperienceramnificationswithregardtoreadinessforsupervisioninternship.

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

PSYC863TeachingandCounselorTrainingRubricPSYC892SyllabusandTeachingPhilosophyRubricPSYC892TeachingPerformanceEvaluationAssessmentComprehensiveExam(CE)Rubric

100%passatPracticelevel100%passatPracticelevel80%passatPracticelevelon1stattempt100%passatMasterylevel

100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionofsecondsemester100%passresultby2ndattempt

Therequirementsforteachingareassessedatmultiplepointsandreadinesstomovefromthetheoryclasstoactualclassroomteachingrequiressufficientmasteryfromtheverybeginning.

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

PSYC605FinalExamPSYC840ResearchDesignProposals(Rubric)PSYC609ResearchProposal(Rubric)

100%ofevaluationscoresindicatepassingatthePracticelevel

100%ofallevaluationsatlevel2,indicatingsufficientmastery,bycompletionof

Studentsweregivenmultipleopportunitieswithinwhichtolearnresearchmethodologyintheoryandinpractice.While

PSYC881LiteratureReview(Rubric)PSYC881ResearchAgenda(Rubric)ProposalandDissertation(Rubrics)

95%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(80%on1stattempt)

secondsemester100%passresultby2ndattempt

somestruggleinthefirstfewcourses,theA&Msupportsystemaswellasmorecounseling-specificlattercourses(application)havehelpedwithaddressingthemasteryneededtodissertationproposalandfinaldissertation.

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

PSYC862AdvocacyinActionAssignmentRubricPSYC810Assignment:EducationSessionRubricComprehensiveExamRubric

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)

100%ofevaluationscoresindicatepassingatthePracticelevel100%ofevaluationscoresindicatepassingatthePracticelevel100%passatMasterylevel(90%on1stattempt)

Studentstypicallyscoreinourdesiredrangeregardingthisdomain.Wedoneedtodeterminewaystobringthetheoryand“proposed”advocacytomoreactivepracticeinthisarea.Talkingaboutversusdoingisthegoalforourstudentsandrevisitingtheassessmenttoolsisontheagenda.

KPI6.Knowledgeandskillsforeffectivecrisis,emergency,anddisasterresponse.

PSYC866CrisisPresentation 100%ofallevaluationsatlevel2practicelevel

100%ofevaluationscoresindicatepassingatthePracticelevel

Thisisanareawehaveidentifiedasaspecialtyforourprogram;however,wecurrentlyonlyofferonecourseandassessatonepoint.Weareengagedindiscussionaboutwayswecansolidifyitasa

truespecialtyareawithinthecurrentprogramstructure.

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).

ProgressReviews(form/rubric)&currentCVevidencingtheirprofessionaldevelopmentbybecomingleaderswhoadvancethecounselingprofessionthroughservice,research,innovation,advocacy,andtraining

100%satisfactoryafteranynecessaryremediationplan(onepersonexitedtheprogramandanotherisabouttoenterremediation).

100%ofevaluationscoresindicatelevel2(mastery)

Thisprocessisconsistentlyidentifiedasastrengthforourprogram.Givenitsdevelopmentalnature,wecannotexpect100%fromeverystudent–particularlyintheirfirstyear.ButwedoexpectagradualimprovementbothinthequantitativeandqualitativeassessmentaswellasacontinuousgrowthpatternsintheevidenceoftheirworkintheCV.

ResultsDissemination(APTElement5)Animmenseamountoftime,energy,andresourcesarededicatedtoconductinghigh-qualityassessment.Fartoomanyresourcesarededicatedtoassessmentforresultstogounused.Thefirststepofusingresultsistoshareresultswithkeystakeholders,suchasfaculty,departmentheads,students,amongothers.Thus,programsshouldclearlyarticulateaplanforresultdissemination.CACREP,requiresthefollowing,under“EvaluationoftheProgram”:

Counseloreducationprogramfacultydisseminateanannualreportthatincludes,byprogramlevel,(1)asummaryoftheprogramevaluationresults,(2)subsequentprogrammodifications,and(3)anyothersubstantialprogramchanges.Thereportispublishedontheprogramwebsiteinaneasilyaccessiblelocation,andstudentscurrentlyintheprogram,programfaculty,institutionaladministrators,andpersonnelincooperatingagencies(e.g.,employers,sitesupervisors)arenotifiedthatthereportisavailable.

Therefore,themajorresultsfromtheAPTandoursystematicassessmentprocesswillbemadeaccessibleonourprogramwebsite,inadditiontootherdatawecollectforourCACREPreport,suchassiteevaluationinformation,exitsurveydata,andemploymentandgraduationrates.Ourreportwillalsobesharedwiththecounselingprogramfaculty,ourdepartmenthead,andmadeavailabletostudentsandothercounselingconstituentsviaourprogramwebsite.Finally,wedrawfromourpreviousyear’sAPTinordertocreateourCACREPAnnualReport,dueeachFall,sharedwithourstakeholders,postedonthewebsite(https://psyc.jmu.edu/counseling/supervision/index.html),anddocumentedforCACREP.TheinformationinthisreportwillbeusedtoinformtheFall2021postingonourwebsite.UseofResultsforLearning&DevelopmentalImprovement(APTElement6A)Ideally,programsuseassessmentresultstomakecurricularandpedagogicalchangesthattheybelievewillassiststudentsinbettermeetingthestudentlearningobjectives.Programsshouldspecifyaclear,detailedplanforusingresults.Inthisplan,programsmayconsiderdescribingtheplannedcurricularchangesbasedonresults,implementationstrategies,implementationdates,andwhy/howthecurricularchangesareexpectedtoimprovestudentlearning.InreflectionofthecurrenteraandCOVIDpandemic,theprogram’sleadership/assessmentcommitteehasbeenfocusingonimprovingthedeliveryofonlinetrainingtocounselinggraduatestudents.AccordingtothecounselingprogramsOnlineLearningAgreementthatwasdisseminatedtoallenrolledcounselingstudents:

Asweworktomaintaintheacademicrigorandintegrityofcounselortraininginadistanceenvironment,werecognizeeveryonewillneedtoadjustandadapttoanewwayoflearning.

PertheCACREP2016standards,counselingprogramshavebeenevaluatingandincorporatingthefollowingstandardsintoourcourseworkandclinicaltraining: Technology’simpactonthecounselingprofession(F.1.j.).

Modalitiesofclinicalsupervisionandtheuseoftechnology(B.2.g.).Ethicalandculturallyrelevantstrategiesforestablishingandmaintainingin-personandtechnology-assistedrelationships(F.5.d).

Overthepastyear,we’veexaminedwaystoimproveourassessmentofonlinelearningintheCounseling&Supervisionprogram.Thechangesthatwe’vemade/aremakinginclude:

1. Studenttelementalhealthcounselingtraining:allstudentsbeforepracticumandinternshipplacementsbeginmustparticipateandcompleteanonlinetrainingmoduleonthedeliveryoftelementalhealthservices.CompletionofthetrainingisuploadedtoourHUBmodule.Thisalignswithstandard(F.1.j.).

2. AdditionofSupervisionAssist:StudentsarenowabletousetheHIPAAplatformforclass,practicaandinternshipexperiences.Thisincludesrecordingsessionswithclientsandparticipationinindividualsupervision.ThisalignswithstandardsF.1.j.,B.2.g.andF.5.d.

• AssessmentofSupervisionAssistcanbeobtainedbycollectingfrequencydataofusers.

• WearestilllearningthenuancesoftheplatformandarestillrelyingoninformationcollectedviaCanvasandourProgressReviewsforassessment.Thereareafewkinksspecifictothediverseclinicalexperiencesofdoctoralstudents.

3. FacultyParticipatedinOnlineTrainingOpportunities:Priortothebeginningofthefall2020semester,allcounselingprogramfacultyagreedtocompleteindividualizedtrainingonteachingonline.

• Wecanuseteachingevaluationsandanecdotalinformationtohelpusassessthis.

4. AdditionofCANVASasalearningmanagementtoolinallcounselingclasses:Priortothebeginningofthefall2020semester,allcounselingprogramfacultydevelopedaCANVASpagefortheirassignedclasses

• UseofCANVAScanbeobtainedbycollectingfrequencydataofusers.

Animportantconsiderationwithour2019-2020APTalternativewastheoverallprogramchangesanticipatedinthePhDprogram.Thisisdifferentthanourothertwoprograms(SchoolCounselingandClinicalMentalHealthCounseling).ThePhDprogramismovingtowardalow-residencydeliverymodelwithcoursesbeingofferedinablendedsynchronous/asynchronousmodelaccompaniedbytwo“residency”weekendsbeginninginFall2021.ThisisdependentonapprovalofaSubstantiveChangeRequestwithCACREP.So,inadditiontotheitemsmentionedabovewhichareconsistentforallthreeprograms,wehaveamassivemigrationandcoursedevelopmentarchitecturaldrivethiscomingacademicyear.Thismayhavesomeimpactonassessment;however,itisdifficulttofullanticipateorarticulatethatfullyatthispoint.

Objective Changeincurriculum Anticipatedtimelineforimplementation Reason(s)forchange

KPI1.Knowledgeandskillsforeffective,evidence-based,culturallycompetentcounseling.

Wearecurrentlycollectingdatafromconstituentstohelpinformourcurriculum,andininformalconsultationwithinternalandexternalstakeholders,weareaddingaseriesofworkshopsdesignedtoenhanceourstudents’preparationforclinicalpracticeincommunitysettings.OneofthecaveatstotheSubstantiveChangeRequestwithCACREPisthatwenotmakesubstantivechangestothecurriculum.Wearehavingtocalibratechangesindeliveryversus“normal”curricularchangesbasedondemonstratedfeedback.

Fall2021 We’rebasingthischangeonourobservationofemergingresearchinthefieldaswellasfeedbackfromgraduatesregardingrecentchangesandtrendsinthefield.FeedbackfromsitesupervisorsandrequirementsofCACREParethereasonweareengaginginadditionalstreamliningconversationswiththesites;although,wedoremaininconstantconversationwiththemthroughouttheacademicyear.

KPI2.Knowledgeandskillsforeffectiveclinicalsupervision.

Nospecificchangebeyondmodifyingsyllabitoincludeemergingresearchaswellasadditionaltrainingindistanceortelehealthsupervisionpractices.

CurrentandFall2021. Workshopswillhelpalignthetrainingthatthemaster’sstudentsaredoingwiththesupervisioninsightneededbythedoctoralstudents.Theywillalsobeinclusiveoftelehealthsupervisionandtraining.Telehealthsupervisionandtrainingunitshavebeenupdatedinthesupervisionandsupervisioninternshipcoursestoreflectcurrentresearch.

KPI3.Knowledgeandskillsrelevanttoeffectiveteaching.

Nospecificchangebeyondmodifyingsyllabitoincludeemergingresearch.Thispastacademicyearwechangedthewaydoctoralstudentsengageintheirfirstteachingexperience.Theynowpartnerwithafacultymemberforanentiresemester.Wehavefoundthisgreatlyassiststheminpreparationfortheirteachinginternship.Needtoexpandourpedagogyrelatedtodistancelearning.

CurrentandFall2021. Continueplacingnewdoctoralstudentswithafacultymembertoco-teachamaster’slevelcourseduringtheirfirstsemester.Wewillcontinuetomonitortheincreasedconfidence,competence,andpreparationfortheirteachinginternship.AdditionalmodulesandpracticeindistancelearningarebeingincorporatedintothePSYC853CounselorEducation(pedagogy)courses.

KPI4.Knowledgeandskillsnecessaryforeffectiveresearchandscholarship.

Nospecificchangeatthistimebutweareconsideringopportunitiesforadditionalresearchcollaboration,coursesectionsdedicatedspecificallytotheneedsourstudents.

CurrentandFall2021. Considerationofchangesisresource-basedandthereforechangesarenotbeingmadeatthistime.WedoneedtoworkmorecloselywithA&MfacultytomakesurewecontinuetobealignedwithCACREPrequirementsintheresearchcourses.TheDeptofGraduatePsychologyisalsoconductingasearchforaresearch-specificfacultymemberwhoshouldbeabletoassistwithraisingthebaronourdoctoralresearchexperiencesandcompetencies.

KPI5.Knowledgeandskillsforeffectiveleadershipandadvocacy.

Moreopportunitiesarebecomingavailableinourfieldfor“emergingleaders”programsinavarietyofprofessionalorganizations.Wewant

Summer2021 We’rebasingthischangeonourobservationofemergingresearchinthefieldaswellasfeedbackfromgraduatesregarding

ourdoctoralstudentstobecomeinvolvedintheseinordertomakeadditionalprofessionalconnectionsanddocumentableexperienceasleadersandadvocates.

recentchangesandtrendsinthefield.Weareaddingopportunitiesforleadershipandadvocacyexperiencestocounttowardinternshiphours.

KPI6.Knowledgeandskillsforeffectivecrisis,emergencyanddisasterresponse.

Nospecificchangeatthistime;however,weareworkingtostrengthenthisareaandaregivingconsiderationtoadditionaloptionsforelectives,service,andtraining.

Spring2022 Spring2022willbringanewinstructortotheCrisis&Emergencycourseandwillincludeclimate-informeddisasterresponse,theneurobiologyoftraumaandACEs,systemicandculturaltrauma,aswellasincorporatingcrisisintosupervisionandteaching.

KPI7.KeyProfessionalCompetenciesandDispositions(asoutlinedintheCounselingProgramsStudentProgressReviews).

Nospecificchangeatthistime.Thishastypicallybeenanareathatouraccreditorshavepraisedasbeinghighlyeffectivesowearenotinclinedtomakeanyadjustmentsatthistime.

InSpring2019,webeganuploadingthesetoaCanvasassessmentsitesobothfacultyandstudentscanhaveaccesstoallpriorProgressReviews.Thisallowsustorefertothemasneededandcompareresultsacrosstime.Wecontinuetoexploreoptionsforusinganothersurveymethodtobettercollect,analyze,andpreserveprogressreviewdata.

UseofResultsforAssessmentImprovement(APTElement6B)Assessmentprocessesmustcontinuallyevolvetoaccommodatenewresearch,changingstudentdemographics,evolvingfaculty/departments,etc.Thus,changeswilllikelybemadetotheassessmentprocessfromyeartoyear.Changesmayincludemodifyingobjectives,changingmeasures,changingwhichstudentsareassessed,changingdatacollectionprocedures,etc.Programsshouldspecifypastchangesaswellasplansforfuturechanges,andmayconsiderprovidingaplanforimplementationoffuturechanges.

Change Anticipatedtimelineforimplementation Reason(s)forchange

WerecentlybegantheprocessofclarifyingourassessmentplaninresponsetoourDecember2018self-studysubmission.Thisyearwehavebeenimplementingtheself-studyandproblem-solvinganyissuesthatemerge.Themajorchangesincluded1)addingprogramspecificKeyCounselorEducationandSupervisionCompetencies(KPIs)toourexistingassessmentplan;2)checkingtheaccuracyofourcurriculummapofobjectiveswithcorrespondingcoursesandexperiences;3)ensuringthateachobjectiveisassessedattheintroductory,practice,andmasterylevelsforallstudents;4)determiningthemostefficientandeffectivemethodforgatheringandmaintainingthesedata;and5)revisingourcommunicationwithexternalstakeholders(alumnae;currentemployers;localpractitioners)toensureongoingapplicabilityofourtrainingtotheneedsofcounselingprogramsacrossthecountry.

OngoingthroughSpring2018

PleasenotethattheassessmentplanwasfinalizedaspartofoursubmissionofourCACREPSelf-StudyinDecember2018.ForFall2018andSpring2019,wecollectedthedataAlthoughwearecurrentlyusingsomeofthesameprocedureswehaveusedinthepasttogatherdataregardingourstudents’progress(comprehensiveexams,practicumandinternshipevaluations,progressreviews),wehavefelttheneedtocreateamoreefficientsystemthatwillallowforeasyaggregationanddisaggregationofdatatobetterenableustoadvisestudentsandnotetrendsacrosstime.Atthispoint,wehavecreatedassessmentsitesonCanvastocollectandshareallKPI-relateddataamongfacultyandwithstudents.AscreenshotofthatpageaswellasascreenshotofoneoftheKPIassignmentrubricsareattachedtothisreport.WearefindingthistobeausefulwaytoassessstudentsusingtheKPIrubricsandallowfortheopportunitytoalsouploadKPIassignmentforreference.Thedataanalysisfunctionwillallowustolookatdataovertimeandacrosscohorts.Wehavealsopurchased(throughstudentfees)aclinicalsupervisionprogramcalledSupervisionAssist.Thisallowsustohouseallclinicalworkforallstudentsincludingclientcounselingvideos,supervisionsessions,connectionstositesupervisors,allcontractsandevaluations.Thisremainsaworkinprogressasweacclimatedtothefullcapacityofthisplatform.Wewelcomefeedbackasweworktostrengthenouroverallassessmentplan.