outcomes · web viewenhanced content related to prescription, application, and fabrication of...

Program OutcomesPhysical Therapy ProgramJuly 26, 2010

The outcomes discussed below are taken from a report written for the Commission for Accreditation of Physical Therapy Education (CAPTE), the accrediting body of physical therapy programs in the US. The report was prepared for a site visit that was held in March, 2010. The entire 230 page review of the program, curriculum, etc, is attached. Information summarized in this document is from sections of the report specifically related to outcomes from the Program.

In the material below, the standard from CAPTE is highlighted in yellow with the response follows.

P-3 The program has expected program outcomes that are based on its goals and reflect the activities of the program, core faculty, and students.

RESPONSE: The PT Program has goals and expected program outcomes that are based on the mission statement, the APTA’s Vision 2020, the Guide to Physical Therapist Practice, and the Normative Model of Physical Therapist Professional Education. From these sources, a curriculum philosophy was developed collectively by the faculty and adopted on June 16, 2004. This curriculum philosophy was the starting point from which the DPT curriculum was developed and all outcome measures are based. The statement of philosophy is elaborated on in Section CP.2 and is described in detail in the PT Student Policies and Procedures Manual (Student P&P) under the Curriculum section.

The expected outcomes, reflecting the program’s goals, have been identified as:

Students/Graduates: 1. Ninety percent (90%) of our graduates will pass the NPTE on their first attempt to

become licensed physical therapists.2. Eight-five percent (85%) of our graduates will feel competent to practice autonomously

as Doctors of Physical Therapy within their first year post-graduation. 3. All (100%) of our graduates will use critical thinking, evidence, and clinical reasoning in

physical therapy patient management. 4. All (100%) of our graduates will be prepared to provide physical therapy care to meet the

needs of patients across the lifespan.

OutcomesCO-1. Graduates of the program meet the expected student outcomes of the program,

including those related to the program’s unique mission.

1

RESPONSE: Graduates of the program meet the expected student outcomes stated in P-3. The extent to which these outcomes are met is analyzed in Table CO-1.1 below. These outcomes are derived from three sources:

Exit surveys: The results derived from exit surveys of graduates are reported as a combined average for the Classes of 2007, 2008, and 2009; all survey results represented a 100% response rate for graduating students.

Graduate Surveys: The results from surveys of graduates 1+ years post-graduation are reported as a combined average for the Classes of 2007 and 2008. The 2009 cohort were not long enough post-graduation to be included in this data set or the results for the employers of our graduates.

Employer surveys: The response rates for graduates and employers were described in Section CP-3 and Table CP-3.2.

Greater than 90% of graduates from all classes met Outcome 1 and Outcome 2. While the results for outcomes 3 and 4 are slightly below our expectation of 100%, it is apparent that the employers recognize that the graduates are prepared. For example, in Outcome 4, the employers indicated that the graduates are prepared to work with patients across the lifespan at a higher percentage rate than the graduates perceive themselves. This can be partially explained by the graduates inability to recognize how their skills will carry over from one population or age group to another in the clinical setting and their lack of experience in multiple settings. This lower response rate can also be explained by the changes faculty have made in the curriculum over the past several years. We expect that this response rate will continue to rise over the next several years as the curricular changes have time to manifest in the more recent, and future, graduates.

In summary, given that our first graduating class was in 2007, the results of the expected student outcomes demonstrate the quality of the program. As the program continues to strive to reach the highest level for each outcome, the input from students, faculty, graduates, employers, and the CAPTE accreditation process will be useful in making changes to influence future outcomes.

Table CO-1.1: Assessment of Program’s Expected Student OutcomesExpected StudentOutcomes

Types of Assessments

Assessment Scoring Method

Results

1) Ninety percent (90%) of our graduates will pass the NPTE on their first attempt to become licensed physical therapists.

National Physical Therapy Licensure Exams (NPTE)

NPTE pass rate and section rates

First time pass rates:Class of 2007: 94%Class of 2008: 93%Class of 2009: 93%

Overall NPTE pass rates:Class of 2007: 100%Class of 2008: 100%Class of 2009: 99%

2

Expected StudentOutcomes



Results

2) Eight-five percent (85%) of our graduates will feel competent to practice autonomously as Doctors of Physical Therapy within their first year post-graduation.

Exit survey of students about to graduate (Classes 2007, 2008, and 2009)

Percent response measured on Likert Scale to online survey questions

92 % agreed the PT Program prepared them to function within a doctoring profession

Survey of graduates 1+ year post-graduation (Classes 2007 and 2008)


86% agreed the PT prepared them to function within a doctoring profession

Survey of employers of graduates 1+ year post-graduation (Classes 2007 and 2008)


92 % agreed the PT Program prepared the graduate with the skills and knowledge necessary for practice as a Doctor of Physical Therapy

3) All (100%) of our graduates will use critical thinking, evidence, and clinical reasoning in physical therapy patient management.



91.6 % agreed the PT Program developed their clinical decision-making abilities

93% agreed the PT Program prepared them to integrate evidence in to clinical practice

94.2% agreed the PT Program prepared them to utilize a critical reasoning process when making ethical and legal decisions regarding physical therapy practice



93.7% agreed the PT Program prepared them to apply a clinical decision-making process

94.9% agreed the PT Program prepared them to apply evidence-based practice

Survey of employers of Percent response 96.2% agree that the

3

graduates 1+ year post-graduation (Classes 2007 and 2008)

measured on Likert Scale to online survey questions

graduate makes appropriate clinical decisions for his/her patients

Performance during final fulltime clinical education experiences

CPI with associated benchmarks for CE III and CE IV

100% of Class of 2009 met entry level on the CPI related to their Clinical Decision making and clinical reasoning skills during their final fulltime clinical internships (CE III & IV)

4

Expected StudentOutcomes



Results

4) All (100%) of our graduates will be prepared to provide physical therapy care to meet the needs of patients across the lifespan.



74.1% agreed that the PT Program prepared them to work with patients across the lifespan



78.4% agreed that the PT Program prepared them to work with patients across the lifespan

Survey of employers of graduates 1+ year post-graduation (Classes 2007 and 2008)


92% agreed that the PT Program graduate demonstrates competence in meeting the health care needs of patients/clients, the local community, and society

CO-2. Graduates of the program meet the health care needs of patients/clients and society through ethical behavior, continued competence, and advocacy for the profession.

RESPONSE: In order to meet the expected student outcomes stated in P-3 and further described in CO-1, graduates of the program will meet the health care needs of patients/clients and society through ethical behavior, continued competence, and advocacy for the profession. It would not be possible to successfully meet the expected outcomes without incorporating all of these practices given that these are recognized components and expectations of a Doctor of Physical Therapy as defined by APTA’s Vision 2020.

CO-3. When averaged over 3 years, 80% or more of all graduates pass the licensure exam.

RESPONSE:

Graduating Class

# who took exam at least

once

# who passed exam after all

attempts

Pass rate per cohort

2007 36 36 100%

2008 43 43 100%

2009 44 43 98%

3-year pass rate: 99%

5

CO-4. Graduation rates and employment rates are consistent with the program mission, goals, and expected student outcomes.

RESPONSE: The 3 year graduation rates are reported in the Graduation Rate Table, and the reported rate of approximately 96% is consistent with the PT Program mission, goals, and expected student outcomes. Given the mission of the PT Program to provide Colorado and the nation with a program of excellence in education, clinical care, research & scholarship, and community service, and the PT Program goal to graduate generalist practitioners in physical therapy who are able to provide physical therapy in the constantly changing health care system, a 100% graduation rate is not likely. The 96% graduation rate does reflect the commitment we have to our students to meet all outcomes and reach graduation while acknowledging that there are changing circumstances that alter the progress or path of small number of students who enter the PT Program.

Survey responses from the graduates of the Class of 2007 and 2008 are reported in table CO-4.1 and indicate that 93% of the graduates sought employment within 6 months of licensure and 91% of them reached that goal. 98% of the graduates who sought employment were employed within the first year. It is not known whether the other 2% sought employment within the first year or made another choice about their career path. While it is inherent in the goal and mission of the PT Program to have 100% of our graduates successfully employed within the first year, personal circumstances occasionally interfere and may delay immediate opportunities for licensure and employment.

6

CP-2.9 A variety of evaluation processes used by faculty to determine whether students have achieved the educational objectives. Evaluations of student performance in the cognitive, psychomotor, and affective domains occur regularly and, at a minimum, must occur at the end of each term of the curriculum.

RESPONSE: Faculty members evaluate students throughout the curriculum to ensure that students demonstrate the level of competence expected at each stage of the curriculum in terms of performing well conceived, safe, and effective physical therapist patient management. These evaluations include assessment in the cognitive domain as demonstrated by performance on written final examinations, assessment in the psychomotor domain (as evidenced by performance on tests of basic skills competencies during most management courses and comprehensive, integrative practical examinations at the end of each management course), and assessment in the affective domain (as evaluated during practical examinations, standardized patients, patient care seminar presentations, and other presentations).

Summative examinations are used throughout the curriculum to evaluate the student’s ability to synthesize content in the cognitive, psychomotor, and affective domains. Courses typically have a final exam and/or final project. Many courses have midterm examinations. Typically the management courses include both assessment of clinical competency and a final practical examination. Also included are presentations in the patient care seminar series (emphasizing cognitive and affective performance). Standardized patient assessments are used at the beginning of the second and third year to assess students’ progress, emphasizing the cognitive and affective domains, but also requiring students to perform psychomotor skills. In addition, several other mechanisms of assessment are utilized including oral presentations in many courses, written papers, development of patient educational materials, and other projects and assignments.

Faculty assess student performance and provide feedback in a timely manner with the intention that students learn from their work. Importantly, students are asked to self assess performance following laboratory examinations, after many of their oral presentations and following many of their group projects. Relative weighting for each assessment for the course grade is indicated in each course syllabus. Students are expected to uphold professional behaviors in all interactions with faculty, staff, and other students in classes, on campus, and during course or curricular assessments. Explanations of specific assessment tools appear in course syllabi located in the appendices to this document. Examples of evaluation methods and graded products will be available on-site for review.

In most courses, students must pass both the cognitive and psychomotor/affective components of the course in order to pass the course as a whole. The PT Program policy on repeating examinations and practical examinations can be found in the Student P&P under the Curriculum section, specifically under the Physical Therapy Program Minimum Grade Standard, Repetition of Courses, and Probationary Status.With regard to clinical education, students receive formal evaluations on their performance in the cognitive, affective, and psychomotor domains at the mid-point and end of each clinical education course. For this purpose, clinical instructors use the Physical Therapist Clinical Performance Instrument (PT CPIWeb). Students complete their own self-assessment separately from the CI using the PT CPIWeb at mid-point and the end of each clinical education course. In addition, students receive ongoing feedback on their performance from their CIs throughout each clinical education experience. During CE III & IV, students and CIs also review monthly Program Benchmarks which are faxed to the

7

DCE/Clinical Education Faculty Advisor for review. Benchmarks are located in the course syllabi for Clinical Education III and Clinical Education IV, in Appendices D in the Clinical Education Manual, and in the Appendices for Section P-11 of this document. No courses in the entry level program are taught through distance education methods.

CP-2.9 A variety of evaluation processes used by faculty to determine whether students have achieved the educational objectives. Evaluations of student performance in the cognitive, psychomotor, and affective domains occur regularly and, at a minimum, must occur at the end of each term of the curriculum.

RESPONSE: Faculty members evaluate students throughout the curriculum to ensure that students demonstrate the level of competence expected at each stage of the curriculum in terms of performing well conceived, safe, and effective physical therapist patient management. These evaluations include assessment in the cognitive domain as demonstrated by performance on written final examinations, assessment in the psychomotor domain (as evidenced by performance on tests of basic skills competencies during most management courses and comprehensive, integrative practical examinations at the end of each management course), and assessment in the affective domain (as evaluated during practical examinations, standardized patients, patient care seminar presentations, and other presentations).

Summative examinations are used throughout the curriculum to evaluate the student’s ability to synthesize content in the cognitive, psychomotor, and affective domains. Courses typically have a final exam and/or final project. Many courses have midterm examinations. Typically the management courses include both assessment of clinical competency and a final practical examination. Also included are presentations in the patient care seminar series (emphasizing cognitive and affective performance). Standardized patient assessments are used at the beginning of the second and third year to assess students’ progress, emphasizing the cognitive and affective domains, but also requiring students to perform psychomotor skills. In addition, several other mechanisms of assessment are utilized including oral presentations in many courses, written papers, development of patient educational materials, and other projects and assignments.

Faculty assess student performance and provide feedback in a timely manner with the intention that students learn from their work. Importantly, students are asked to self assess performance following laboratory examinations, after many of their oral presentations and following many of their group projects. Relative weighting for each assessment for the course grade is indicated in each course syllabus. Students are expected to uphold professional behaviors in all interactions with faculty, staff, and other students in classes, on campus, and during course or curricular assessments. Explanations of specific assessment tools appear in course syllabi located in the appendices to this document. Examples of evaluation methods and graded products will be available on-site for review.

In most courses, students must pass both the cognitive and psychomotor/affective components of the course in order to pass the course as a whole. The PT Program policy on repeating examinations and practical examinations can be found in the Student P&P under the Curriculum section, specifically under the Physical Therapy Program Minimum Grade Standard, Repetition of Courses, and Probationary Status.

8

With regard to clinical education, students receive formal evaluations on their performance in the cognitive, affective, and psychomotor domains at the mid-point and end of each clinical education course. For this purpose, clinical instructors use the Physical Therapist Clinical Performance Instrument (PT CPIWeb). Students complete their own self-assessment separately from the CI using the PT CPIWeb at mid-point and the end of each clinical education course. In addition, students receive ongoing feedback on their performance from their CIs throughout each clinical education experience. During CE III & IV, students and CIs also review monthly Program Benchmarks which are faxed to the DCE/Clinical Education Faculty Advisor for review. Benchmarks are located in the course syllabi for Clinical Education III and Clinical Education IV, in Appendices D in the Clinical Education Manual, and in the Appendices for Section P-11 of this document.

No courses in the entry level program are taught through distance education methods.

Curricular EvaluationCP-3. There is on-going and formal evaluation of the professional curriculum. The

curriculum evaluation plan is written and addresses individual courses within the curriculum, as well as the curriculum plan as a whole. The plan incorporates consideration of the changing roles and responsibilities of the physical therapist practitioner and the dynamic nature of the profession and the health care delivery system. Data are collected from appropriate stakeholders, including, at a minimum, program faculty, current students, graduates of the program, and at least one other stakeholder group such as employers of graduates, consumers of physical therapy services, peers, or other health care professionals. The evaluation plan is used to determine strengths and weaknesses of the curriculum and to determine if the practice expectations and specific mission, goals, and expected student outcomes of the curriculum are met.

RESPONSE: Faculty regularly perform a systematic review of the curriculum (Table CP-3.1). Information is reviewed and analyzed from multiple data and information sources to identify strengths and weaknesses of the educational program and to determine whether the PT Program’s mission, goals and objectives are being met. In addition, ongoing and regular discussions are held as needed at the bi-weekly biweekly faculty meetings. Associated Faculty are encouraged to participate in these meetings. Feedback also is provided from students formally at Director’s Time (beginning of each semester), at committee meetings with class representatives (e.g. Curriculum Committee and Recruitment, Admissions and Matriculation Committee), course evaluation forms completed by students at the end of each course, focus groups with graduating students, input from the surveys of recent graduates and their employers, and national licensure exam scores. In addition, feedback on the curriculum is solicited from clinical instructors during clinic calls and visits during each clinical education experience and from our Core Affiliate Clinical Advisory Group during annual meetings.

Table CP-3.1: Annual review of the curriculum planSource of Data Timeframe Responsible people

Discussions at Curriculum End of each semester Faculty as a whole; Program

9

Review Meetings 2004 - 2009 Director

Program retreats Annually Faculty as a whole; Program Director

Students Beginning of each semester

Class meeting with Program Director

Curriculum Review Grids 2007, 2008 and 2009 Curriculum CommitteeSpecific Course evaluations End of each semester Student evaluations, reviewed by

Course Coordinator and Program Director

Discussion by Core Course Instructors

End of each semester Core Course Faculty

Exit Interviews Graduation week Online survey is completed by graduating students and analyzed by Assistant Program Director and faculty; Focus groups are led by core faculty assisted by administrative staff for note taking

Graduate Surveys One year post graduation Alumni complete the online survey and the Assistant Program Director analyzes the data

Employer Surveys 16 months post graduation Employers of graduates complete the online survey and the Assistant Program Director analyzes the data

Formal Curricular Reviews: The curriculum is reviewed in its entirety at annual faculty retreats. The communication among faculty during these discussions has helped to integrate information and build on previously presented content across courses and has assured faculty that the learning objectives of each course are appropriate for the curricular sequence. The faculty use the Normative Model of Physical Therapist Professional Education1 to assess course objectives and descriptions.

From 2005-2008 (the first three years of the new curriculum) the faculty met for one day at the end of each semester to review the curriculum (minutes of these meetings are available on request). These meetings were held to assure that portions of the curriculum, taught for the first time, met with expectations and intended outcomes. They were also held to assure that all threads were indeed threaded throughout the curriculum and to review specific content areas. To this end, at each meeting the discussion focused on the new curriculum for the semester that was just completed, the performance of the curriculum as a whole, and a review of a specific content area and/or threads. As examples, the musculoskeletal content was reviewed in Spring 2006; Psychosocial Aspects of Care was reviewed in Spring 2007. Adjustments were made as needed to specific content areas as well as overall curriculum organization. Between 2005 and 2008, the curriculum was reviewed as a whole at the end of each year. Beginning this year, meetings are now held each fall to review the curriculum as a whole.

1 American Physical Therapy Association (APTA). Normative Model of Physical Therapist Professional Education: Version 2004.

10

Sources of data used in these curricular reviews include the following: Student course evaluations Faculty assessment of performance in the course and in subsequent courses that build

on that content Performance on the licensure exam (NPTE) Student discussions with Program Director each semester (Director’s time) Exit interviews Graduate surveys Employers surveys

Minutes from the review meetings and the documents reviewed for each thread and/or content area reviewed will be available for on-site review. The changing roles and responsibilities of the physical therapist and the health care delivery system are considered in curriculum assessment as they relate to the educational program objectives and individual course objectives as well as learning activities and student assessment strategies. This process is facilitated by the fact that a number of faculty members (see Table F-2) maintain an active clinical practice, hence they are able to keep the PT Program as a whole apprised of changes that should be reflected in the curriculum. The ongoing assessment of program objectives and expected student outcomes assures us that the curriculum reflects current best practice and all professional values and documents.

Information is obtained from a variety of stakeholders during the curriculum evaluation of the curriculum. Included are students, graduates of the PT Program, employers of the graduates, as well as the faculty. Performance of the graduates on the licensure examination (Table CO-3) is also useful, as well as performance on specific sections of the examination.

Student evaluation of the curriculum occurs at various points within their educational program. Students complete course evaluations at the conclusion of every semester. They also meet with the PT Program Director at the beginning of every semester (‘Director’s Time’). The information obtained from these two sources was particularly helpful during the first few years of the implementation of the DPT curriculum and was used to refine both course content and placement within the curriculum.

Graduate (alumni) surveys for the classes of 2007 and 2008: The PT Program began to collect data from each cohort of graduates approximately one year after their graduation and licensure. These Program graduates provide a first-hand perspective on how well the curriculum prepared them for their initial entry into practice as a physical therapist. The first cohort to evaluate the curriculum was the Class of 2007. As alumni, these graduates were contacted and asked to complete a survey regarding the curriculum during the fall of 2008. This timing was significant in that we wanted to survey them after most had likely been employed as physical therapists for at least one year. Twenty-five of 36 graduates completed the survey for a return rate of 69%. The process was then repeated in early fall 2009 to capture the responses from the graduates of the Class of 2008 after they had been working as physical therapists for about 1 year. The response rate was 77% with 33 of 43 graduates from the program completing the survey. Given the challenges of contacting graduates once they leave the program, and often leave the metropolitan area or state, these response rates are well-representative of the cohort.

Questions were developed to gain an understanding of how well the curriculum addressed

11

content areas and threads. Generally, the alumni indicated overall satisfaction with the curriculum and their preparation as physical therapists entering the profession. The courses provided a foundation for critical thinking and problem solving and prepared the graduates of the PT Program to promote health and wellness in individual patients and populations, practice safely, work with a variety of patient types and diagnosis, and function as a doctor of physical therapy and member of the healthcare team, recognized as an expert in movement related function. A summary of survey results is in Table CP-3.2 and several areas of concern are described in more detail in CP-3.4. The complete survey results will be available on-site.

Table CP-3.2: Summary of results of the Graduate SurveysSurvey Question Percent of

Respondents to Agree and

Strongly Agree – Class 2007

Percent of Respondents to Agree and


Total Respondents =

25/36 (69%)

Total Respondents = 33/43 (77%)

The Physical Therapy courses provided me with a foundation for critical thinking and problem solving.

90 % 100 %

The PT Program prepared my communication skills in these areas:- Verbal- Non-verbal- Written

806550

968989

The PT Program prepared me to value service learning. 55 71The PT Program prepared me to promote health and wellness with individual patients/clients.

85 89

The PT Program prepared me to promote health and wellness with patient populations.

80 86

The PT Program prepared me to understand and apply business-related concepts to patient care delivery in the clinical setting.

65 50

Survey Question Percent of Respondents to

Agree and Strongly Agree

– Class 2007

Percent of Respondents to Agree and


The PT Program prepared me to appropriately utilize support personnel (PT Assistants, Aides, etc.) within the clinical setting.

50 39

The PT Program prepared me to consider myself as the health care team member recognized for expertise in movement analysis and the application of movement related to function.

90 89

The PT Program prepared me for my first position as a physical therapist.

75 85

The PT Program prepared me to function within a 80 86

12

doctoring profession.The PT Program prepared me to provide safe clinical practice.

95 100

The PT Program prepared me to work with patients across the lifespan.

65 71

The PT Program prepared me to work with many types of patients (e.g. musculoskeletal, neurological, medical).

65 82

The PT Program prepared me to work with populations (e.g. groups of people who share similar diagnoses or have similar health-related needs).

80 89

The PT Program included the necessary basic and applied sciences (histology, physiology, anatomy, movement science, etc.) that have allowed me to make appropriate clinical decisions.

85 96

Employer surveys for the graduating classes of 2007 and 2008 were contacted to complete a survey after the graduates returned the information letting us know where they were employed. The ability to obtain information through a survey of employers required that the graduates respond to our request about their employers and provided us with accurate information to contact the employer. Overall, the results from thse employers indicated that they are highly satisfied with the graduates from this program. Responses were received from 12 of the 25 employers provided by the graduates of the Class of 2007. This represents 48% of the employers for these graduates or 33% of the total employers for all graduates of the Class of 2007. Responses were received from 13 of the 33 employers provided by the graduates of the Class of 2008. This represents of 39% of the employers of graduates for these graduates or 30% of the total employers for all graduates of the Class of 2008. Findings from these surveys are summarized in Table CP-3.3 with full results available on-site.

Table CP-3.3: Summary of results from the Employer SurveysSurvey Question Percent of

Respondents to Agree and


Percent of Respondents to

Agree and Strongly Agree

– Class 2008Total

Respondents = 12/25 (48%)

Total Respondents =

13/33 (39%)The graduate makes appropriate clinical decisions for his/her patients

100 88

The graduate was well prepared to begin practice as a physical therapist compared with others PTs they have employed or worked with in the past

100 88

The graduate practices in an ethical and legal manner and the graduate demonstrates competence in meeting the health care needs of patients/clients and society

100 100

The University of Colorado Denver Physical Therapy Program prepared the graduate with the skills and knowledge necessary for practice as a Doctor of Physical Therapy

92 88

13

These data were supported by comments such as, “The graduate at my facility has consistently exceeded my expectations for her level of experience. She is highly motivated, seeks out additional responsibilities, and her clinical skills are outstanding. She has focused on the development of a specialty program at our facility and while managing a high caseload keeps on track with marketing this program and practicing evidence-based practice in all areas.”

Summaries of evaluation materials, including course evaluations, graduate and employer surveys, and student exit interviews, will be available to review on-site.

Curriculum Strengths All curriculum review data are used during the annual Curriculum Review Meetings. Curriculum Strengths that were identified by the faculty in 2008 and noted again in 2009 included the following:

Length of clinical education Life span philosophy Generalist approach Patient centered focus Clinical reasoning focus Ability of students to integrate evidence into reasoning Use of instructional technology in classes Emphasis on cultural competence Collaboration among faculty

o Integration across the curriculumo Collaborative teaching in classeso Collaborative activities and assignments to develop and implement threads and

overarching frameworks Community volunteer program Use of standardized patients Student centered program Rates of success on NPTE Number of faculty with diverse academic backgrounds and perspectives

Areas that could be strengthened:The curriculum review data also indicated the need for several changes in the program. Areas needing improvement were identified in 2008, and several changes were made. These areas needing attention included:

Placement of several courses in the curriculumo Educational Methodso Patient Care Seminar III

Limited emphasis on specific content in the curriculumo Prosthetics and orthoticso Integumento Airway clearance techniqueso Utilization of support personnel

Inconsistency across syllabi organization and lack of clarity of course objectives Inconsistency in grading and providing feedback on writing assignments

14

Based on the above, we took action in late 2008 and in 2009 to make curricular changes and address these areas needing improvement. Actions taken to remediate are as follows:

Moved Educational Methods from Spring 2 to Fall 1 Moved Patient Care Seminar III Spring 3 to Fall 3 Enhanced course content in the Medical Conditions I and II to better address integument

and airway clearance techniques Enhanced course content in Medical Conditions II to better address the GI/GU system Enhanced course content in Health Promotions to address the need for women’s health

related to GI/GU system Revised content of Applied Exercise Science to better address exercise progression Revised course content in Health Care Delivery to include information on utilization of

support personnel including supervision and delegation in the clinical setting Enhanced content related to prescription, application, and fabrication of devices and

equipment in patient management courses, in addition to what is covered in Prosthetics and Orthotics

Adjustments to PCS course assignments to address timing with other workload expectations

Reorganized the Neuromuscular II and III and Musculoskeletal Conditions III course content to insure pediatric content within the lifespan approach

Enhanced course content in Radiology to incorporate imaging that focuses on neuromuscular and medical conditions – previously focused primarily on the musculoskeletal system

Improved consistency of syllabi using a template Improved clarity of objectives for courses Modified Director’s Time to be a more effective means of communication Developed writing rubrics for consistent guidelines in written assignments across the

curriculum

A few areas continue to need attention. These were identified during the Curriculum Review Meeting of Fall 2009 as part of our ongoing curriculum evaluation:

Consider the need to better balance workload in first year of programo Consider moving a course from Summer I to Fall or Spring I

Defensible Documentation o Consider additional opportunities for practice in courses

Ability of students to progress plan of care across continuum (goal setting and projecting anticipated change in acute care and rehabilitation), including discharge planning

Experience with delegation and supervision of support personnel (PTAs and unlicensed personnel)

CP-4. There is ongoing and formal evaluation of the clinical education program.

RESPONSE: The clinical education program is evaluated on a regular basis with input from multiple sources including the clinical education team, core faculty and Program Director, core affiliate clinical advisory group, clinical education faculty (CIs and CCCEs), and students. Information is collected and analyzed annually on components of the clinical education program using the following Outcome Tools/Assessment Methods:

Clinical Performance Instrument Monthly Benchmarks for Clinical Education III and IV Course Assignments during Clinical Education Experiences

15

Benchmarks/Recommended Qualifications of Clinical Instructors APTA Student Evaluation of Clinical Education Experience and Instruction Forms Student Exit Interviews Student Exit Interview Focus Groups Annual Commitment Forms for internship offers Clinical Site Information Forms Student and CI/CCCE surveys Information gathered during site visits and calls (forms) Discussions during meetings with the Core Affiliate Clinical Advisory Group Annual Review with Program Director Discussions during Clinical Education Team Meetings

Samples of these tools can be found in the Appendices. Completed tools and meeting minutes will be available on-site. Proposed changes based on the ongoing and annual reviews are presented to the PT Program Director during monthly meetings with DCE and Core faculty during biweekly faculty meetings.

A summary of evaluation results and changes made along with plans for 2009-10 follow on subsequent pages.

Placement of Clinical Education in the Curriculum. Based on feedback from de-briefing meetings and exit interviews with students, meetings with core affiliate advisory group, and clinical education team meetings, the placement of clinical education in the curriculum is appropriate. No changes have been made and no changes are planned.

Length of Clinical Education in the Curriculum. Input from several individuals (students, CCCEs, CIs, Clinical Education Team, Core faculty and Program Director, and an outside consultant) was used in 2006 to evaluate the length of clinical education in the new DPT curriculum. Feedback indicated that six weeks was too long for CE I, since students had limited knowledge and skills after just two semesters in the curriculum. In addition, students could benefit from additional time in order to more fully meet the goals and expectations during CE II. Based on this information, the length of Clinical Education I was shortened from six weeks to four weeks and the length of Clinical Education II was lengthened from six weeks to eight weeks. The length of Clinical Education III and IV was lengthened from 12 weeks to 16 weeks as a precursor to considering the possibility of incorporating a six-month or year-long internship, as well as providing students more time to practice with autonomy. Feedback from the Classes of 2008 and 2009 on the Exit Surveys and informal discussions with CIs during clinic calls and visits about the increased length of CE III and IV is not conclusive. A survey of CIs and CCCEs will be conducted in 2010 to formally analyze the effectiveness of the changes made in the length of CE III and CE IV. Given these changes, the overall length (44 weeks) of fulltime clinical education appears adequate for students to achieve the Expected Student Outcomes.

Practice in Clinical Education Sites. Overall feedback about clinical sites has been positive. Students report they are able to practice patient-centered care. Information gathered during meetings with students (de-briefing sessions after Fieldwork I and CE II), review of course assignments related to outcome measures (Health Care Delivery) and evidence-based practice (Scientific Inquiry III), and review of Item 9 on the APTA Student Evaluations of Clinical Education Experience related to professional practice and growth indicate that areas needing to be improved are a) consistent application of evidence-based practice and b) outcome measures in clinical practice. In addition, Program information sent to sites prior to each clinical education experience does not always reach the students’ specific CI/s. Changes that have been

16

implemented include: 1) creation of the “Clinical Instructor’s Community,” a web-based resource site for CIs and CCCEs; 2) creation of pod-cast learning modules for CIs and CCCEs on EBP and using outcomes in the clinic; 3) increased use of electronic communication with CIs and CCCEs to share Program expectations for each clinical education experience; and 4) increased coaching with students to take responsibility for sharing Program information (in addition to Program sending ahead of time). Plans for 2009-2010: 1) create pod-cast learning modules for CIs and CCCEs regarding preparation for taking a student for each clinical education experience (fall 2009 – spring 2010); 2) improve accessibility of Clinical Instructors Community by creating a common user name and password for all users; 3) host a Content Update Course on EBP for CIs and CCCEs (spring or summer 2010), and 4) host the Advanced CI Education and Credentialing course annually (content includes EBP and review of the elements of the Patient/Client Management from The Guide to Physical Therapist Practice).

Number and Variety of Clinical Sites/Need for Additional Sites. Currently the PT Program has ~280 active clinical sites. In 2008-2009, 311 internship slots were offered, 25 slots were canceled, and 177 slots were used. The PT Program added 25 new sites during this time period: 13 out patient orthopedic; 6 pediatric; 4 in patient rehabilitation; 2 hospital; and 1 home health. Students perceive there are not enough sites available, however all students are able to meet Program requirements and to gain the type of experiences in which they are interested. In 2008, the Clinical Education Team implemented individual meetings with clinical education advisees in the spring 2 semester to identify the types of sites and experiences needed for Clinical Education III and IV. This has helped the team to focus recruitment on specific sites/experiences needed to ensure that adequate number of sites are available to students making selections. In addition, this has prevented recruiting excessive slots that will not be used. Plans for 2009-10: Although the number of clinical sites was adequate in 2008-09, the Clinical Education Team anticipates the need to increase the number of active clinical sites approximately 10 to 15% in 2009-10 (~30-45 sites). The increased number of sites will accommodate the increased number of students in the class of 2012. Also, the plans recruit more Acute Care and Rehab/Neuro sites (as as not to over-utilize/tax our established sites in these areas) and specialty practice sites ( e.g. Pediatric & Advanced Ortho /Sports sites) to respond to student’s requests for internships in these settings. In order to do so, as well as to accommodate local trends in healthcare that have affected clinical education (e.g., staffing changes, increased productivity demands, etc. that have led to more internship slot cancellations and decreased number of commitments), an additional faculty member was added to the Clinical Education Team (25% FTE) in 2009-10 to assist with developing new sites and meeting other team responsibilities.

Clinical Education Documents/Assessment Tools. Documents, forms, and assessment tools are evaluated by the Clinical Education Team on an annual basis. Based on information gathered during clinic calls and visits, meetings with the core affiliate clinical advisory group, and Clinical Education Team meetings, feedback about the PT Program’s clinical education documents has been positive. The Clinical Education Manual was converted to an electronic document in 2008-09, which has been more economical and easier to distribute, making it more accessible to students, Center Coordinators of Clinical Education (CCCEs), and CIs. Center Coordinators of Clinical Education and CIs also have the option to request a hard copy or CD-ROM, although very few requests have been made. The PT Program implemented the use of the CPIWeb in 2008-09. The greatest challenge for CIs has been accessing the APTA training and CPIWeb initially. Once users are in the system, the tool is easy to use. The new CPIWeb has enabled the Clinical Education Team to better track student progress, more clearly identify whether students have met Program’s criteria for “Passing,” assess accuracy of the CIs’ evaluation of student performance, and facilitate communication between the Team and

17

CIs/CCCEs. The DCE and Administrative Assistant created a pod-cast learning module to assist CIs, CCCEs, and students in accessing the CPIWeb. Clinical Education Forms (such as commitment forms, commitment verification forms, Benchmarks for CE III and CE IV, request to open a new site, etc.) have facilitated communication and contributed to smooth operations in the Clinical Education Program. The majority of tools used to assess the Clinical Education Program provide qualitative data. In 2008-09, two assessment tools were created to provide more comprehensive, quantitative data: 1) a database that tracks aggregate CI and site data from the APTA Student Evaluation of the Clinical Experience and Instruction (to assess quality of student experiences and CIs) and 2) a formal survey to assess the effectiveness of the DCE and Clinical Education Team. These tools coupled with the qualitative assessment tools used provide a more complete data set about the effectiveness of the Clinical Education Program.

Quality of Clinical Education Faculty. The quality of Clinical Education Faculty is assessed using the APTA Student Evaluation of the Clinical Experience and Instruction and by the Clinical Education Team during clinic calls and visits and reviewing CI comments in the CPIWeb. These are effective mechanisms. Results can be found in detail in Sections F-23 and 24.

Clinical Faculty Development Plan. The clinical faculty development plan is designed to enhance communication between the academic program and clinical sites and to meet the needs of clinical faculty based on the DCE’s analysis of information gathered during clinic calls and site visits, Student Evaluations and meetings, discussion with the Clinical Education Team, and input from the Core Affiliate Advisory Group. The plan is evaluated on an annual basis.

Clinical Faculty needs for development as identified in 2008/09: Increase number of credentialed clinical instructors Better understanding and implementation of evidence based practice Development of site objectives/plan for clinical education experiences Strategies to communicate expectations and student concerns clearly and early on in the

experience Use of the CPIWeb

The DCE also identified trends related to the CCCE: Increasing numbers of CCCEs are not physical therapists, and therefore do not have a

clear understanding of physical therapy education There is high turnover in the CCCE position, and therefore many do not have a clear

understanding of the important and multifaceted role of the CCCE CCCEs have limited time available for developing a structured clinical education

program and for mentoring CIs and students

Clinical Faculty Development Activities in 2008-09: Taught 6 APTA CI Education and Credentialing Programs

o 4 basic level courses (April, 08; August, 08; October 08; October 09)o 2 advanced level courses (October 08 and May 09)

Hosted 3 Content Update Courseso 2 Ortho, including best evidence and suggested outcome measures (July 08;

May 09)o 1 Neuro, including best evidence and suggested outcome measures (Sept 09)

Hosted 1 discussion group for PTs in acute care with faculty content expert and Colleen Kigin PT, DPT, FAPTA, national expert (April 09)

Increased accessibility of the Clinical Instructors’ Community by creating a common user

18

name and password for all users Created 3 pod-cast learning modules for the Clinical Instructors Community

o Accessing and using the CPIo Preparing for students in Clinical Education III & IVo Preparing for students in Clinical Education I

Co-hosted regional Clinical Education Conference with Regis University and sponsored with Northwest Intermountain Consortium (September 09)

o Juggling the roles of the CCCEo Facilitating clinical reasoning in the clinical settingo APTA regional forum on standards in clinical education

Plans for 2009-10 include: 1) Promote and track the use of Clinical Instructors Community; 2) create CI learning modules on preparing to mentor students (CE III & IV completed in August 2009; CE I completed in December 2009; plans underway for CE II); 3) increase communication/support for CIs during each clinical education experiences; 4) continue outreach to CIs working in acute care to facilitate the process of defining the role of the PT in acute care; 5) conduct meetings with CIs from similar areas of clinical practice (e.g., pediatric, SNF, acute care, OP Orthopedics) to discuss/define “entry level” practice and facilitate the development of objectives for clinical education experiences; 6) provide update course on Evidence Based Practice in spring or summer 2010; and 7) Dr. Jody Gandy will assist the Program with developing a strategic plan for the Clinical Education Program (including clinical faculty development) in April 2010.

Clinical Education Resources. Clinical Education resources are evaluated by the Clinical Education Team and Program Director. Evaluation in 2008-09 indicates: 1) the budget is adequate to support the current Clinical Education Program activities (separate from salary support); 2) there is a need to expand the Clinical Education Team in order to enhance the quantity and quality of the clinical education program; and 3) the PT Program will benefit from expanding the Core Affiliate Advisory Group. Changes made based on evaluation in 2008-09: 1) The Clinical Education Team was able to keep costs low for all CIs and provide financial assistance to 5 CIs to attend clinical faculty development activities; 2) Program Director approved DCE’s proposal to add a faculty member (25% FTE) and administrative support (additional 25% FTE); and 3) five additional members were added to the Core Affiliate Advisory Group representing different practice settings and patient populations, an invaluable resource that guides the Clinical Education Team in carrying out the activities of the Clinical Education Program.

As mentioned above, Dr. Jody Gandy has been hired as a consultant and will assist the Program in developing a three year strategic plan for the clinical education program. The clinical education team, Program Director, Assistant Program Director, core faculty, CCCEs, CIs, managers, and employers will participate in this process as well.

19