1

2014/15 Student Learning Assessment Report

October 2015

2

2014/2015 MHA Student Learning Assessment Report

Introduction The Master of Health Care Administration (MHA) program updated its Student Learning and Assessment Plan (SLAP) for the 2014/2015 academic year (AY) taking into consideration feedback provided by the Student Learning and Assessment Committee (SLAC), the program’s faculty, as well as changes the program has undertaken through D2L curriculum mapping. The most significant change has been that each course is now mapped to the objective and assessment levels rather than just the course level. With the assistance of Dr. Devrim Ozdemir and D2L Insights reports, we now have several direct measures of student learning in addition to some indirect measures from our graduate exit survey. This report also includes a map of how our courses support the DMU university-wide learning outcomes. A plan for continuous improvement in measuring learning outcomes, and how the program can use this information to enhance student learning is provided.

MHA Learning & Program Objectives

The MHA program uses the National Center for Healthcare Leadership (NCHL) competency model. As stated by NCHL (n.d.), this model “was developed to provide the field with a comprehensive, validated competency model that will be suitable as the foundation for a breadth of leadership assessment and development applications. To ensure relevance to leaders from across disciplines, the Model was developed and validated utilizing interdisciplinary subject matter experts, and was refined in collaboration with industrial and educational psychologists. The resulting model has been adapted for use extensively in healthcare settings, and is now the leading model in use by accredited graduate programs in healthcare management, according to research conducted by CAHME” (http://www.nchl.org/static.asp?path=2852,3238). The MHA program is well into the process of obtaining Commission on the Accreditation of Healthcare Education (CAHME) accreditation. The NCHL model is the most widely used model with CAHME accredited programs hence, this model is a good fit for the program. The NCHL model consists of 26 competency categories in three domains of transformation, execution, and people as shown in diagram below. Within each category, multiple levels of competency exist for a total of 123 individual competencies.

3

4

Learning Objectives: All MHA courses are mapped to all three domains and all 26-competency categories; however, not every individual (123) competency is mapped or addressed in the MHA curriculum. All categories are assessed and monitored for improvement using the programs annual course evaluation process.

While all of the NCHL competencies are important, the faculty has focused on nine key competency categories (three from each domain) to track and report the program’s learning outcomes since 2010.

a. Financial Skills b. Strategic Orientation c. Communication d. Collaboration e. Accountability f. Interpersonal Understanding g. Analytical Thinking h. Professionalism i. Self Development

Program Objectives: In addition to reporting student learning on 9 key competency categories, the MHA programs Operations Plan (2012-2015) outlines two additional objectives to be included in this report:

1. Students will successfully complete the culminating experiences of the curriculum: Field-Based Learning or Capstone course (depending on when student enrolled in the MHA program) as measured by completion rates and cumulative GPA. 2. MHA graduates will be satisfied with their academic investment at DMU/MHA as measured by the Graduate Survey.

Data Collection and Outcomes 1. Direct Measures: As a result of curriculum mapping to the course objective and assessment level, direct measures were obtained through D2L Insights reporting for the nine key competencies. Example of rubric used in courses is included below. There are several; this is simply an example used for discussion boards. Each rubric is linked with an assessment and objective.

5

6

7

It is important to note that this report shows the program course-level learning outcomes for a limited “snapshot in time.” These average percentages reflect the average percent of students who attained a particular competency based on the corresponding course objectives during the 2014/15 academic year. Most competencies are mapped to more than one course objective, so each time a student is evaluated on a competency the achievement (met or not met) will update or change. Students not achieving a competency in one course may achieve it later in another course. Students performed close to 100% on accountability, collaboration, interpersonal understanding, and professionalism. Students will have an opportunity to hone communication skills in the final residency. Analytical thinking peaks during Field Based Learning (MHA 749), which is the final course in the program. Self-development continues throughout the program with an emphasis during Professional Development Seminars scheduled within the three on campus residencies (MHA 801-803) with advising and career development activities. It is encouraging to see that strategic orientation is above 90%, as this was identified as a weakness in past reports. The addition of a strategy course (MHA 742) appears to have had a significant impact. Financial skills was also identified as a challenge in the past. The lead faculty for the finance courses (MHA 625 and 630) continues to refine these courses with added tutorials, synchronous test prep sessions on-campus or via Adobe Connect, use of teaching assistants, changing assessment methods, etc. Despite these efforts, overall student achievement is below our desired goal. The course lead continues to research teaching/learning methods to raise student financial skills competency. Action planned: Ultimately, it is our goal for all students to achieve 100% competency in all areas. Going forward, D2L Insights reports can be run for each student. Faculty advisors will use this information to identify areas where students are falling short and advise them on ways to achieve competency. This could be through seeking out external mentors or planning a field based learning project to address the shortfall. As this is new information for us, we will continue to refine the reports, set benchmarks, and create processes to ensure our students are achieving their full potential. In addition, we plan to run reports on graduating students that will more reflective of student learning outcomes at the end of the program rather than a snapshot course learning outcomes. This will give us more accurate data on culminating learning outcomes. 2. Mapping to DMU university-wide learning outcomes: The MHA program strongly supports DMU learning outcomes. Action plan: MHA will continue to support DMU and will report on all five each academic year to monitor student achievement for success and opportunities for improvement.

8

9

3. Indirect Measures: The following chart includes historical data on primarily indirect measures, including action plans.

Objective

MeasureToolDirect/Indirect

Bench-mark

ActualDataandDateCollected

Whousesthedata?

NarrativeSummary

Action(s)Planned/TakenthatAddressCurricular

Changes

.

1.ProgramObjective:Capstone/FieldBasedLearning:StudentswillsuccessfullycompletetheculminatingcourseintheMHAcurriculum

Comparisonofenrollmentwithcompletion.(Direct)

90%completerate(internal)

Year CompletionRate2015 100%2014 97%2013 95%2012 95%2011 98%2010 72%2009 77%

Program

Goalmet.33/34complete.

As“legacy”studentsgraduate,the2credithourCapstonewillbephasedoutandreplacedbythe3credithourFieldBasedLearning.Itprovidesamorerigorousandstructuredexperiencetofullyintegratethecurriculumintoaculminatingexperience.ThecourseleadforFBL(MHA749)continuestomakecourseimprovementbasedonstudentfeedbackandinstructorexperience.Primarily,thefocushasbeenonpreparingstudentsbeforetheyenrollinthecourseonappropriate

10

projects/preceptors/hostorganizationsandhowtowritearesearchpaper.

GPAofgraduatingstudents(Indirect)

Median3.0orabove(internal)

Year

GPA

2015 3.72014 3.682013 3.752012 3.53

Program Met.MHAstudentshavenoboardexam,sothisisincludedasanindicatorofstudentsuccessintheprogram.

Rigorofprogramisevaluatedthroughtheindividualcourseevaluationreports(CER’s)andstudentperformanceonthelearningobjectivesviaD2LInsights.Theprogramiscurrentlyintheprocessofextendingtheexistingcurriculummaptomeasurehigh/lowerteaching/learningmethodsandassessmentmethods.Oncecompleted,thisinformationwillinformtheprogramcurriculumcommitteeontheacademicrigor.Asaresult,theculminatingGPAmayexperiencedecline.

2.ProgramObjective:Percentageofstudentscompletingthegraduatesurvey.

GraduateExitSurvey(Indirect)

Responserate>80%(internal)

Year ResponseRate2015 82%2014 89%2013 91%2012 77%2011 38%

Program Met.37/45-Responseratedecreasedbutbenchmarkmet.

Continuetoincorporatethegraduatesurveyintotheexitingprocesstoachieveourdesiredresponserate.

3.aDidtheprogramhelpyoufurtherdevelopcompetencyinHealth

GraduateSurvey(Indirect)

80%agree(internal)

Year PercentAgree2015 89%2014 91%

Program Met,althoughthepercentagedroppedabit.Maybedueto

Continuetoeducatestudentsonthe

11

CareAdministration? 2013 94%2012 100%2011 100%

higherresponserate. competencymodelduringthefirstresidencyandthroughtheportfolioassignment.Createdacloserconnectionbetweenthecourse,assessmentsandcompetenciesusingtheMHAsyllabustemplate.

3.b.Didthecoursesofferedintheprogramadequatelyprepareyouforyourdesiredcareer?

GraduateSurvey(Indirect)

80%Agree(internal)

Year PercentAgree2015 91%2014 91%2013 96%2012 97%2011 93%

Program Met,consistentwithpreviousyears.

Studentsarebettereducatedonhealthcarereformandtheuncertainlythatliesahead,soaslightdropistobeexpected.Seeadditionalcommentsfor3.d.oncareerpreparedness.

3.c.DidyourexperienceintheMHAprogramhelpachieveyourgoal?

GraduateSurvey(Indirect)

80%Agree(internal)

Year PercentAgree2015 94%2014 87%2013 94%2012 100%2011 100%

Program Met,goodtoseeanincrease.

Willcontinuetoeducatestudentsforthefutureandenhancecareeradvising.Seeadditionalcommentsfor3.d.oncareerpreparedness.

3.d.Didtheprogramhelpyoudevelopacareerplanorprovidecareerdirection?

GraduateSurvey(Indirect)

80%agree(internal)

Year PercentAgree2015 79%2014 81%2013 79%2012 81%2011 80%

Program NotMet.Thishasbeenconsistentlylow,butweanticipateanupwardtrendastheadvisingandleadershipplanshaveplayedamoreprominentroleinresidencies.

Acareerpaneldinnerwasdevelopedforthethreerequiredresidencies,whichhasreceivedverypositivefeedback.Additionaltimewasaddedtotheresidencyforadvisorstoreviewtheiradvisee’sleadershipdevelopmentplansandcareergoals.Programfacultyarealsoresearchingacurriculumchangethatwouldintegrateformal

12

careerplanning/strategycontentthroughtheresidencyexperience.Thischange/additiontothecurriculumhasbeennotedinotherCAHMEMHAprograms.

Pleaseprovideanswerstothefollowingquestions:

1. Haveyoumappedyourcurriculumcompetencies?Yes,thecurriculumhasbeenmappedthreetimes.ThefirsttimewasApril2010whenthecurriculumwasmappedatthecourseleveltotheNCHLcompetenciesandBloom’sTaxonomy.Teachingmethodsandassessmentswerereportedforeachcourse.Thesecondtimewasin2012,atthecourselevel,usingaformwithintheCAHMEcandidacyapplication.Further,withtheadditionofDr.DevrimOzdemirandourtransitiontoDesire2Learn,allcourseshavebeenmappedtothecourseobjectiveandassessmentlevel.

2. Ifso,atwhatlevel(e.g.course,objective,assessment,etc.)?Itismappedtotheobjectivelevel.Wearealsoexaminingeachcoursetoensurethathigher-levelteaching/learningmethodsarebeingusedwithhigher-levelassessmentsbasedonBloomstaxonomy.Thiswillbecompletedbytheendofthe2015/16academicyear.Further,severallowerlevelassessments(suchasquizzes)wereremovedorarenolongergraded;theyareforstudentsformativelearning.Eachsummativeassessmentineachcourseisattachedtoarubrictoprovidedirectmeasuresforcompetencyachievement(met/notmet).Wehavealsoremovedstudentself-reportsontheNCHLcompetenciesasameasureoflearningoutcomes.Wesupportthatself-assessmentisvaluableforstudentreflectionandoverallleadershipdevelopment,butresultswillnotbereportedhere.

3. Whattooldidyouusetoconductthemapping?Doyouhaveanyfeedbackonthetoolutilized?Desire2Learn.TheprocesswaslaboriousandwecouldnothavedoneitwithouttheguidanceofDr.Ozdemir.However,nowthatitisdone,wecanaccesstheInsightsLearningAnalyticReportsandgleanusefuldatatohelpimproveourcourses.

Summary: The focus of this report is to identify and implement opportunities for curricular changes to improve student learning. This is the first time we have been able to pull direct learning outcomes from D2L. As noted above, these data provide only a snapshot of learning outcomes in November 2015 for the 2014/15 Academic year (Terms included: Fall 2014, Spring 2015 and Summer 2015). Students who did not meet a learning outcome at this time may achieve it later in the program. Ultimately, it is our goal for all students to achieve 100% competency in all areas. Going forward, D2L Insights reports can be run for each student. Faculty advisors will use this information to identify areas where students are falling short and advise them on ways to achieve competency. This could be through

13

seeking out external mentors or planning a field based learning project to address the shortfall. As this is new information for us, we will continue to refine the reports, set benchmarks, and create processes to ensure our students are achieving their full potential. In addition, we plan to run reports on graduating students that will more reflective of student learning outcomes at the end of the program rather than a course-learning snapshot during a students program of study. This will give us more accurate data on culminating learning outcomes. Going forward, the focus on key competencies will change. The faculty reviewed prior Student Learning Assessment Reports, competency achievement data and considered feedback from the MHA Advisory Board and decided on the following for 2016. We may have the opportunity to drill down even deeper into the levels of achievement in each competency.

1. Analytic Thinking

2. Communication Skills

3. Impact and Influence

4. Innovative Thinking

5. Professionalism

6. Relationship Building

7. Strategic Orientation

8. Team Leadership

9. Change Leadership

Regarding the indirect measures, plans are noted in each section. Overall students are performing well with completing the Field Based Learning course and graduating from the MHA program with strong GPAs. The graduate exit survey indicated that career planning was not as robust as desired. Therefore, we will be enhancing this area with advising and researching career planning/strategy curricular enhancements for the residencies. This new content would integrate existing content/exercises (i.e. career strategy [MHA 742], networking opportunities [MHA 801-803]), the development of leadership development plans [MHA 801-803] and the development of career goals [MHA 801-803] while adding new content on career planning, resume writing, conducting a job search, interview technics, social media tips, and changing the learning portfolio [MHA 801-803] to a career portfolio. In addition,

14

beginning in 2014, the program is surveying alumni (six months from program completion) to glean more information on career progression after graduation. Overall, the new curriculum that was implemented in Fall 2012 appears to be improving student performance in key areas although the full extent of improvement will take several years to fully appreciate as our “legacy students” who enrolled prior to 2012 graduate and newer students have experienced the entire program. Additional reports from D2L will give us a more accurate view of the learning outcomes of graduates. The MHA program supports University wide learning outcomes and will continue to do so. All faculty members are actively engaged in the evaluation of learning outcomes and curricular improvement. We employ a continuous quality improvement process that involves all major stakeholders. Improvements are made on a continuous basis as feedback from students, faculty, and other stakeholders (such as the MHA Advisory Board, employers, and DMU administration) indicate the need. Further, the impact of the rapidly changing health care system is integrated into courseworkinatimelyfashion.

15

The diagram below is a snapshot of the student-centered approach we have adopted: