outcomes assessment 101

Outcomes Assessment 101

Assessment Showcase 2009:

Best Practices in the Assessment of Learning and Teaching

February 11, 2009

Gwendolyn Johnson, Ph.D.Deborah Levine-Donnerstein, Ph.D.Assessment Coordinating Council

Agenda for Today

• Benefits of Assessment

• Procedure for Assessing Student Learning

• Learning Outcomes Statements

• Commonly Used Measures of Student Learning

“Assessment is a means for focusing our collective attention, examining our assumptions, and creating a shared academic culture dedicated to continuously improving the quality of higher learning. Assessment requires making expectations and standards for quality explicit and public; systematically gathering evidence on how well performance matches those expectations and standards; analyzing and interpreting the evidence; and using the resulting information to document, explain and improve performance.”

Angelo, Thomas, “Reassessing Assessment,” AAHE Bulletin, Volume 47/Number 8, April, 1995. This definition of assessment was adapted by the University of Arizona.

Definition of Assessment

Effective Assessment

Adapted from work of T. A. Angelo – University of Miami – June 1998

1. Assesses what matters most – in what we actually teach and in what we expect our students to learn

2. Provides information for improving learning (and for improving teaching and assessment)

3. Focuses on process as well as on inputs and outcomes

4. Actively involves teachers and students (and appropriate others)

5. Uses multiple and varied measures

6. Is carried out at various key points

7. Provides feedback to those most affected

(B) Review statements from professional organization; peer institutions

(D) Investigate existing measures not currently in use

(G) Identify relevant externally defined criteria (e.g. what do peers do?)

(1) Determine desired learning outcomes

(B) Prioritize outcomes; decide on a schedule for assessing particular outcomes

(2) Select multiple measures for each outcome being measured

(3) Carry out assessment; tally and analyze results

Set standards; Benchmarks

(4b) Use results to fine-tune the assessment process

(4a) Use results to improve program

(C) Find out what is already being assessed; revise/adapt as needed.

(A) Get input from stakeholders: faculty, students, alumni, employers, parents, etc.

(E) Develop measures as needed

(F) Determine sample size, schedule of administration, etc

A Procedure for Assessing Learning Outcomes

OIRPS Johnson 9/5/08

Adapted from work by E. Berman 1999

Learning Outcome Statements

• Describe what students are able to demonstrate in terms of knowledge, skills, and attitudes upon completion of the program

• Rely on concrete verbs such as “define,” “apply,” or “analyze”

• Map to the curriculum and educational practices that offer multiple and varied opportunities for students to learn

• Ideally, are collaboratively authored and collectively accepted

• Incorporate or adapt professional organizations’ outcomes statements if they exist

• Can be quantitatively and/or qualitatively assessed during students’ undergraduate or graduate studies

Collectively created and developed across the program, learning outcome statements:

Learning Outcome Statements

Students will be able to:

Distinguish and articulate major cultural differences between the target culture and their own.

• Analyze how specific historical and cultural contexts shape particular perceptions, practices, and products of individuals

• Interpret mathematical models and represent information symbolically, visually, numerically, and verbally.

• Write persuasive documents that provide appropriate and effective evidence for various audiences, situations, and purposes.

• Demonstrate knowledge of the formal and informal structures and processes that make social systems, governments, and economies work.

• Design experiments, generate and analyze actual data, use abstract reasoning to interpret these, and formulate and test hypotheses with scientific rigor.



(H) Identify relevant externally defined criteria (e.g. what do peers do?)





(G & H) Set standards; Benchmarks







(G) Consider level of incoming students with regard to benchmark

A Procedure for Assessing Student Outcomes



Commonly Used Measures of Student Learning

Direct Indicators of Learning

• Pre/Post Tests

• Course -Embedded Assessment (papers, assignments, tests)

• Standardized Tests (national, accreditation)

• Portfolios

• Capstone Courses

• Thesis/Dissertation Evaluation

• Observation of Performance

Indirect Indicators of Learning

•Student surveys

-questionnaires -interviews and focus groups

• External Reviewers

• Alumni surveys

• Employer Surveys

• Curriculum and Syllabus Analysis

Commonly Used Measures of Student Learning















(G) Consider level of incoming students with regard to benchmark




A Program Assessment Example

Unit Student papers/

projects

Student portfolios

Embedded assignments

/testing

Capstone

Course evaluation

Outcomes

Externally evaluated

HIS 101 x x x x

HIS 200 x x x x

HIS 210 x x

HIS 309 x x x x

HIS 322 x x x

HIS 400 x x x

HIS 410 x x x

HIS 450 x x x x

The Important Questions to Consider:

Do the student learning outcomes match current-established goals?

Are the current outcomes what we intend to measure and assess?

Overall, are students acquiring our expected: skills, knowledge, attitudes and motivation to succeed in their academic experiences?

What changes are needed to address gaps in assessing student learning?

How can we proceed to identify shortfalls between curricula and learning outcome goals at each level?










(4a) Use results to improve processes





(G) Consider level of incoming students corresponding to benchmarks




• How can the assessment process be sustained?

• Can efficiency in collecting, reporting, and using results be increased in all relevant areas of campus?

• Should we change the standards, criteria, benchmarks, etc., on an ongoing basis to meet internal and external constraints facing universities?

• In what ways must campus policy support and reflect assessment goals for student learning?

General Considerations

Sustainability:

Assessing Learning Outcomes in a Changing Academic Environment

• Within the process of restructuring colleges, programs and departments, a campus retains leadership and gains new leaders to:

continue to enhance the goals of assessing student learning,

reshape and modify procedures for assessing student performance throughout this process,

reestablish linkages among new and existing units to better inform:

• what is important to assess, • how to address changes in programs and curricula,• develop effective measures to assess new goals and• determine how best to use these outcomes in the classroom,

department and college for greater student retention and success

References

Angelo, Thomas. “Reassessing Assessment,” AAHE Bulletin, Volume 47/Number 8, April, 1995.

Maki, Peggy L. (2004). “Assessing for Learning: Building a Sustainable Commitment Across the Institution”. Sterling, VA: Stylus Publishing

Walvoord, Barbara (2004). “Assessment Clear and Simple”. San Francisco, CA: Jossey-Bass

http://www.provost.wisc.edu/assessment/manual/manual2.html

Gwendolyn Johnson, Ph.D.

Director of Assessment, Office of Institutional Research and Evaluation

Chair, Assessment Coordinating Council

[email protected]

520 621-7726

http://assessment.arizona.edu

Deborah Levine-Donnerstein

Department of Educational Psychology

Contact Information

outcomes assessment 101

Documents