Outcome Assessment Reporting for Undergraduate Programs

Stefani Dawn and Bill BogleyOffice of Academic Programs, Assessment & AccreditationFaculty Senate, April 11, 2013

Assessment is…

• “…the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning.”

– B. Walvoord, Assessment Clear and Simple, 2004

Assessment 1-2-3

1. OUTCOMES (“Students will be able to…”)2. EVIDENCE– Direct measures (exams, papers, performances…)– Indirect measures (surveys, placement…)– Qualitative and quantitative indicators

3. IMPROVEMENT & VALIDATION– What does success look like?– How do we know?

How well are students interpreting and responding to our expectations for their learning?

Assessment at OSU: Highlights

1997• Assessment of learning in Student Affairs

20002005

• Assessment for professional accreditation• Undergraduate reports (BEAVERS Learning Outcome

Tracking System)

20112012

• Graduate program and Bacc Core assessment• On the right track, keep going (NWCCU)

NOW: Next Steps for UndergraduateProgram Assessment

• Revised Assessment Reporting Template– Clarify what is needed; provide guidance &

examples– Reinforce the fundamental role of program

outcomes– Endorsed by Executive Committee, Curriculum

Council, University Assessment Council• Initiate institution-level feedback & synthesis

New Assessment Report Template

• Features– Spreadsheet file format– Tabs: Introduction; Instructions; Example; Report

Year; Add new years over time– In the report: The information that needs to be aligned with outcomes is aligned in a single table

New Assessment Report Template

• Process Clarifications– Each year one or more of the program outcomes must

be measured such that ALL outcomes have been assessed in a period of 5 years. • A plan must be in place to do this – the plan is simply

communicated in the report table in this reporting document.

– There needs to be at least 1 direct assessment per outcome. • If an outcome only has indirect assessment, then an

explanation is requested.

New Assessment Report Template

• Process Clarifications– As programs develop their assessment

infrastructure, we encourage solid, in-depth assessment of 1 or 2 outcomes; do it well rather than trying to assess too many outcomes at once.

– Timeline:• This template will be used for the September 15, 2013

submission.

3. Program Outcomes Matched with Measures and Results

Question Example

List your program level student learning (SLO)

Design, implement, monitor, and adjust an individualized, evidence-based, pharmaceutical care plan that will maximize patient safety and therapeutic efficacy.

Is this SLO/SSO new or revised since the last year you reported on it?

No

What year will you be reporting on this outcome?

2012

List the measures/methods/instruments used to assess the outcome.

Pharmaceutical Care Lab Clinical Cases to develop a care plan (3rd year); Advanced Pharmacy Practice Experience (APPE) Preceptor Evaluation on specific care plan elements (4th year)

Are the measure(s)/methods/instruments direct (D) or indirect (I)?

D

What benchmarks are you using to determine if the outcome has been satisfactorily met by the students?

Clinical cases - 95% of the class will receive at least a 75% score on the first attempt; APPE - 95% of the students will receive at least a 70% on each of the element scores

3. Program Outcomes Matched with Measures and Results

Question Example

What do the data show about student learning for each outcome?

100% of the students passed both direct assessments at the defined benchmark. This is an improvement from 92% last year for the clinical cases and 90% for the PPEs.

Course level changes Last year we implemented course level changes in the Pharmaceutical Care Lab to provide students with more opportunities to practice writing and receive feedback on their Care Plans. The success of this change is indicated in the student success rate - nothing else was changed (assessments, rubrics, and PPEs remained the same, students were just provided more opportunities for practice and receive feedback).

Program/degree level changes None at this point

3. Program Outcomes Matched with Measures and Results

Question Example

Process unit used to reflect on the data The typical process is that the data collected are presented to the curriculum committee, who then identifies and prioritizes action items and responsible parties. Those action items are shared with the dean's executive committee who presents questions to the curriculum committee, approves the action items and/or allocates resources as needed.

Assessment plans for the upcoming year We plan to assess learning outcome #4 next year. Although this outcome is assessed each year in the course, because we are focusing on this program level assessment of the outcome, the course instructors will meet and develop the specific exam questions for the assessment and then have the questions reviewed by three faculty from the curriculum committee to ensure the questions map well to the outcome and are clearly written and understandable. For open-ended questions, the results will be graded by two faculty and scores averaged to represent the student score for those test items. The results will be presented to the curriculum committee.

• Over the next two months and/or summer:• Consultations/feedback on last year’s reports• Orientations to the new form

• Available Support• Addressing specific feedback items• Preparing for the new report template• Targeted consultations – e.g. direct vs. indirect assessments,

matching outcomes with assessments, organizational process ideas

• 1-on-1 meetings, special sessions at departmental meetings or retreats, etc.

Report Feedback & Unit Meetings

Questions?

• Stefani.Dawn@oregonstate.edu (7-0919)• Bill.Bogley@oregonstate.edu (7-3506)