using portfolios for program assessment

Using Portfolios for Program Assessment

Sakai ConferenceMay 31, 2006

Judy Patton, Candyce Reynolds, and Wende Morgaine

Portland State University

Agenda

E-portfolio for institutional assessment and accreditation

E-portfolio for program assessmentE-portfolio as a pedagogical tool for

learningMoving from web-based portfolio

development to OSP

Assessment Practices

Assessment Serving at least 3 goals:

Institutional---------Programmatic---------Classroom/Pedagogical

The nature of interpretation, meaning of the assessment changes as a function of the audience and the goal of the assessment.

Portland State University

Urban university Doctoral intensive Seven colleges/schools – Arts and Sciences,

Urban and Public Affairs, Business, Fine and Performing Arts, Engineering and Computer Science, Grad School of Social Work, Grad School of Education

24,000 students 80% undergraduate

Key Influences – Research on own students

High percentage of first generation students Broad range of prior academic experiences/preparation High percentage of returning/adult students High proportion of students who began college work

elsewhere Majority of students commute/ no on-campus “place” Majority of students work half-time or more/ off-campus Large number who attend part-time and take more than

6 years to graduate

PSU Assessment Cycle

Using Portfolio Thinking for Accountability and Improvement

Electronic institutional portfolios initiated with Urban Universities Portfolio Project

UUPP had reform of accreditation self-study process as a goal

Institutional portfolios allow institutions to show multiple examples of their work and offer a view of development over time

Web provides a medium for showing the interconnectedness of institutional work

www.portfolio.pdx.edu

Table 1. Basic principles of the biological sciences (see text) as met by individual courses.

Biology Courses * 1 2 3 4 5 6 7 8

101-103 Gen Bio/lab X X X X X X X X

175 Evol Concepts X X X

252,253 Princ Bio X X X X

301, 302, 303 Hum AP X X X X

336 Cell Biology** X X X X

338 Intro Molec Bio X X X X X X

341 Genetics** X

357 Gen Ecology** X X X

387 Vert Zoo X X X X X

412 Animal Behav X X X X X

Student e-Portfolios

Purpose of University Studies - PSU

The purpose of the general education program at PSU is to facilitate the acquisition of knowledge, abilities, and attitudes which will form a foundation for lifelong learning among its students.

PSU University StudiesPSU’s 4 level general education programFour goals:

Inquiry and Critical Thinking Communication Diversity of Human Experience Ethical and Social Responsibility

UNIVERSITY STUDIES REQUIREMENTS

45 credits

credit

15

12

12

6

TT210

FRESHMAN INQUIRY

UPPER DIVISION CLUSTER

SENIOR CAPSTONE

SOPHOMORE INQUIRY

1 2 3

1 2 3

Why Portfolios?

The best evidence of student learning is found in actual student work.

A tool for organizing, assessing, and displaying student work is the student portfolio.

Benefits of the Portfolio

Portfolio-based assessment systems are built upon organized collections of actual student work.

Portfolios provide a means to evaluate student progress and development as well as achievement.

Portfolios provide a means for all students including those with different learning challenges to document and display their achievements.

Portfolios provide an opportunity for students to document their learning both in the classroom and in their participation in outside activities.

Why Portfolios?

Portfolios contribute to student learning.Electronic portfolios are collections of

digitally represented artifacts that: Document practice, Include reflection, Integrate experience, Map to goals and/or standards.

Key Performances:

“A key performance is the work (evidence) a student submits to demonstrate progress toward or achievement of a learning goal.”

For example: a research paper, exam, a creative work, a taped oral presentation, a business plan, or the results of an experiment.

Includes supporting material such as student self-reflection, peer review, and faculty comment.

Rubrics: Rubrics are scoring systems which define the

evidence (see “Key Performance”) needed to demonstrate achievement of particular learning goals set by the major and/ or the institution.

Rubrics are diagnostic (not just the student’s best work) allowing us to pinpoint student progress (or lack thereof) and achievement.

E-Portfolio Benefits: Electronic vs. Hard-Copy Hyperlinks – (encourage metacognition). Multiple Goals (critical thinking, writing, visual

communication, group work). Curricular and Extra-curricular—includes

learning which occurs within, between, and outside the classroom.

Technical Skills.

E-Portfolio Benefits: Electronic vs. Hard-Copy Scholarship as Public enterprise. Public Assessment. Audience Consideration. Students Working with Students – peer tutoring

and interaction.

Main point

Value on student work and voiceEmphasis on using assessment as a

means to give student a sense of their capacity

Most importantly, assessment serving the need of student learning.

Benefits to Students:

Students understand and can articulate what they have learned.

Students can organize and display key performances and other material to demonstrate what they know and can do.

Digital medium allow a variety of kinds of work samples (i.e. art, oral presentation, performances…).

Benefits to Students:

Digital structure gives students the flexibility to organize their presentation portfolios in different ways for different audiences.

Provides students with the opportunity to communicate their accomplishments to graduate programs and employers in ways that give meaning to the transcript and grade point average.

Assessment of Portfolios

Freshman Inquiry Rubrics Locally developed six point rubrics for 5 areas Average

freshman would be expected to score at Levels 2- 3 Portfolio review process

Faculty across campus serve on committee 1 rubric per day Norming/calibration 2 raters for each assignment 3Rd rater if scores do not match

Summative evaluation for program Formative assessment for faculty teaching teams

PSU Student Portfolio Project:

Four level e-portfolio. Organized around “key performances”. Student progress and achievement determined through

the application of scoring systems or rubrics. Developmental: Learning assessed through actual

student work over time. Diagnostic: Areas of adequate/ inadequate progress can

be targeted and specifically dealt with. Curricular, Co-curricular, and Extra-curricular: Includes

student learning which occurs within, between, and outside the classroom.

Closing the Loop

Data from eportfolio review is shared with faculty teams

Teams are asked to discuss the results and determine what, if any, changes will be made in the curriculum

Reports are sent back to the program administration

Each year, program assessment committee studies annual assessment results and creates a new plan based on current questions.

Portfolio Assignment

Centered around the 4 goals of UNSTGeneral Reflection essayReflection on each goalStudents provide evidence of achievement

in the goal area80% of portfolios are now electronic

Assessment Process

Faculty Responsibility to attach the classroom activities to the larger program goals

Random Sample of Portfolios are gathered (n=150)

Faculty in UNST and in other departments rate achievement using rubrics

Team report Program report Part of University assessment process

Rubrics

Writing (part of Communication goal)Quantitative Literacy (part of

Communication goal)Ethics & Social ResponsibilityDiversity of Human ExperienceCritical ThinkingAll Rubrics—6 point scale

Comparison of Rubric scores

0

0.5

1

1.5

2

2.5

3

3.5

4

writing ethics diversity critical QL

2001

2002

2003

Team Differences

Team Differences

Findings and Responses

Consistency of scores Want Diversity scores to be higher

Faculty Development in area of diversity Teams differed

Team Reports allow reflection and ability to address how they will improve their curriculum in the next year

Learning from each other Quantitative Literacy Diversity

Faculty Development is a key component of improving student learning

Examples