Assessing Student and Institutional Learning Using

Electronic Portfolios

Trudy BantaSharon Hamilton

Susan Kahn

The Assessment Institute in Indianapolis

November 3, 2003

Our Questions:

How can electronic portfolios be used to assess

1. student learning?

2. institutional learning and effectiveness?

What have you accomplished

in the last 4 years?

Did you have any difficulty summarizing your accomplishments?

What would have helped you do this?

How would you organize

a presentation on student learning

on your campus?

Is there a conceptual framework

that you could use?

The Search for Accurate Measures of Student Learning

1980 – Estimated gain in Tennessee

1992 – Federal proposal for national test

2000 – Report Card (Incomplete in learning)

2002 – New proposals for assessing

learning

ePort at IUPUI

Connecting and Assessing Curricular and Co-Curricular Learning

What is ePort

• A collection of student work throughout their academic career at IUPUI, selected to show learning of the Principles of Undergraduate Learning in relation to course work.

• An opportunity for students to develop their own learning profile and learning matrix, knowledge maps, advising record, and resumes.

Authn/Authz Security WorkflowComm.Tools

Storage

“Unbundling”

In Touch

Syllabus

Testing

Gradebooks

Schedule

EreservesDigital content

Full Text articles

Federated Searching

Learning ProfileLearning Matrix

Advising

Resume/Vita BuilderResearch Manager

Content Manager

Knowledge Mapper

Authn/Authz Security WorkflowCommTools.

StorageOther

ServicesOther

Services

ePortfolio Goals• Assist both faculty and students to reach a

clearer, more coherent understanding of how aspects of the curriculum support students' increasing mastery of the PULs.

• Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution.

• Support student engagement with the PULs over their entire undergraduate experience, beginning in the freshman learning community and culminating in the capstone experience.

ePortfolio

The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning.

1. Core communication and quantitative skills2. Critical thinking3. Integration and application of knowledge4. Intellectual depth, breadth, and

adaptiveness5. Understanding society and culture6. Values and ethics

ePortfolio Levels of Competence

– Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours.

– Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours.

– Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program.

– Experiential: Connecting curricular and co-curricular learning

QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.

Complete Pending Ready Locked

Learning Matrix

View by Category View by Matrix

Click on cell to view/edit

Core Communications & Quantitative SkillsWritten CommunicationAnalyzing TextsOral CommunicationQuantitative Problem SolvingInformation Literacy

Learning Matrix

View by Category View by Matrix

Click on cell to view/edit

Core Skills > Oral Communication

Persuasive Speech

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Add this to another Cell.

1. a. b. c.

Course Development Project

PosterOne Page document

2.

Learning Matrix

View by Category View by Matrix

Click on cell to view/edit

Core Skills > Oral Communication > Introductory

Students communicate orally with different audiences and purposes.

To demonstrate your understanding of this PUL at the Introductory level, the oral communication and documents you upload and your reflection should show the following:

a) you identify the characteristics of your intended audience and adapt your speech to this assessment and analysis

b) you create a specific purpose or function for your oral communicationc) you organize and develop your main ideas to accomplish your purpose for

your intended audience;d) you locate and identify credible sources of information on your topice) you identify appropriate communication aids (for example, power point slides,

handouts, audio tape or video tape) that will aid in accomplishing your specific purpose

f) you use effective oral (rate, volume, pauses, articulation) and nonverbal (eye contact and body language) delivery skills

g) you identify and understand critical listening skillsh) you identify your own and others strengths in oral communication, which

includes strengths in organization, content, delivery and audience adaptation

Learning Matrix

View by Category View by Matrix

Click on cell to view/edit

Core Skills > Oral Communication > Introductory

Students communicate orally with different audiences and purposes.

To demonstrate your understanding of this PUL at the Introductory level, the oral communication and documents you upload and your reflection should show the following:

a) you identify the characteristics of your intended audience and adapt your speech to this assessment and analysis

b) you create a specific purpose or function for your oral communicationc) you organize and develop your main ideas to accomplish your purpose for

your intended audience;d) you locate and identify credible sources of information on your topice) you identify appropriate communication aids (for example, power point slides,

handouts, audio tape or video tape) that will aid in accomplishing your specific purpose

f) you use effective oral (rate, volume, pauses, articulation) and nonverbal (eye contact and body language) delivery skills

g) you identify and understand critical listening skillsh) you identify your own and others strengths in oral communication, which

includes strengths in organization, content, delivery and audience adaptation

Audience: These are the intended viewers/ listeners of your oral communication. Your intended viewer/listener may be one person or a group, which will require you to consider who they are and what their perspectives are on the topic. The oral communication you choose should demonstrate how you can adapt your speaking to your viewers/listeners.

Learning Matrix

View by Category View by Matrix

Core Communication & Quantitative Skill

Critical Thinking

Integration & Application of Knowledge

Intellectual Breadth, Depth and Adaptiveness

Understanding Society and Culture

Values and Ethics

Learning Matrix

View by Category View by Matrix

Core Communication & Quantitative Skill

Critical Thinking

Integration & Application of Knowledge

Intellectual Breadth, Depth and Adaptiveness

Understanding Society and Culture

Values and Ethics

Written CommunicationAnalyzing TextsOral communicationQuantitative Problem SolvingInformation Literacy

Introductory Intermediate Advanced Experiential All

Can we assess institutional

learning and effectiveness

using an electronic

institutional portfolio?

Urban Universities Portfolio Project (UUPP)

California State University, Sacramento Georgia State University IUPUI Portland State University University of Illinois at Chicago University of Massachusetts Boston

Sponsor: AAHE Funded by: The Pew Charitable Trusts

(1998-2001)

“Institutional Portfolio”

A focused selection of real work, combined with interpretation and reflection, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness.”

Lessons Learned: Advantages ofElectronic Institutional Portfolios

Facilitate campus involvement Bring new perspectives that catalyze learning

and change Focus on evidence and alignment Demonstrate institutional effectiveness and

improvement over time Demonstrate accountability Useful for multiple purposes and audiences

Lessons Learned: Disadvantages of Electronic Institutional Portfolios

More work than a paper report!Scarcity of modelsNeed for infrastructureAccreditation/accountability in transition

—organizations/teams may need to be oriented to this approach

Blur “boundaries” of self-study/report

Next Steps

Develop new iteration/update as annual performance report

Update the design and technological infrastructure

Incorporate more examples and aggregated evidence of student learning through interface with student electronic portfolio