e-portfolio development (aka e-cobbling) at the university of cincinnati wayne hall (vice provost...

E-Portfolio Development

(aka E-Cobbling)

at the University of Cincinnati

Wayne Hall (Vice Provost for Faculty Development

and Professor of English & Comparative Literature)

Richard Robles (Assistant Director for First Year Experience and

Leadership, University Honors Program)

Presented at the Cohort V meeting of the Inter/National Coalition

for Electronic Portfolio Research

(George Mason University, Fairfax, Virginia, August 2008)

Is this one of those sessions where I have to take a bunch of notes?

The URL that contains all:

• http://homepages.uc.edu/~hallwe/e_cohort.doc

The University of Cincinnati

• Research-intensive university• Liberal-arts plus professional colleges• Tradition and culture of de-centralized colleges• Current enrollment around 36,000, with the goal

of 40,000• A General Education program that began in 2001,

with the first cohort graduating in June 2005

Integrated Core Learning

The phrase “Integrated Core Learning” describes the University of Cincinnati’s signature approach to undergraduate education. ICL was developed as the thoughtful integration of the General Education core, major course work, undergraduate research, practicum, clinical placements, co-op and service-learning experiences, co-curricular activities, and faculty-guided reflection throughout the curriculum

E-portfolios to capture academic and co-curricular experiences

So what kind of e-portfolio program did we want?

College of Applied Science at UC

Honors Program at UC

Another path: E-Cobbling

“The learning portfolio is a flexible, evidence-based tool that engages students in a process of continuous reflection and collaborative analysis of learning. As written text, electronic display, or other creative project, the portfolio captures the scope, richness, and relevance of students’ learning. The portfolio focuses on purposefully and collaboratively selected reflections and evidence for both improvement and assessment of students’ learning” (16).

(Jossey-Bass, 2004)

Helen Barrett, 2003

The “cathedral” model: most of thecommercial world; vs. the “bazaar” model (the Linux world)

Published 2001

The Bb “Grade Center” as a low-tech manager of a student’s

learning portfolio

Through Bb’s “content system,” I can send a folder to be assessed and/or archived.

Bb Content System:

UC template for e-portfolio in Bb

2nd UC template for Bb e-portfolio

Multiple systems for multiple functions (and for a variety

of technology skills & interests)

• An “online assessment system” here, a “Web-sensible” system there– Kathleen Blake Yancey, “Postmodernism,

Palimpsest, and Portfolios: Theoretical Issues in the Representation of Student Work” (2004)

Gardner Campbell, EDUCAUSE Review (July/Aug 2008):

What kind of behavior do you want to reinforce? What kind of community do you want to enable within your environment? Then look around and ask: “Are there lightweight tools, small pieces loosely joined, that we can bring together into a particular environment to support that kind of community?” Maybe the idea is not to buy some big product and deploy it as an enterprise activity but instead to encourage people to innovate and experiment in small, lightweight ways with open source tools, web enabled tools, sometimes even in a kind of ad hoc way depending on the particular course and the particular set of learners in the course.

Lisa Lane, “Toolbox or Trap” (2008)• Web 2.0 applications that encourage social

construction of knowledge (Wiki-spaces, BubbleShare, Ning) are freely available and may provide more creative instructors with better options than any LMS currently available. Such programs make possible the creation of one’s own mini-CMS, cobbled together out of programs that fit with the instructor’s methodology. In these cases, pedagogy comes first – the tools can be used to build the course we want to teach.

What about some recent student examples from the World of Web 2.0?

http://homepages.uc.edu/~engelhcr/sotlhomepage.pdf

ChaleeEngelhardReflections section

http://laugh.as.you.cover.me.with.leaves.googlepages.com/

Sarah Vogt Blackboard discussions:

Sarah Vogt draft with comments and further reflection:

http://maryjotewes.googlepages.com/teachingportfolio

Mary Jo Tewes section on student work from writing classes:

Mary Jo Tewes section on conference presentations:

http://ms.mkramsey.googlepages.com/e-portfolio%2Cscholarshipofteachingandlear

Angela Arndt - Dissertation Wiki

Access rights – Diss. Wiki

Contents of Dissertation Wiki

And now for something completely different:

• Ohio’s version of the Voluntary System of Accountability

The Voluntary System of Accountability (VSA)

• Result of a partnership between the American Association of State Colleges and University (AASCU) and the National Association of State Universities and Land-Grant Colleges (NASULGC)

• Approved in autumn 2007• http://www.voluntarysystem.org

The VSA’s “College Portrait” contains sections for

• Student and Family Information• Student Experiences and Perceptions• Student Learning Outcomes

– Measure of Academic Proficiency and Progress (MAPP)

– Collegiate Assessment of Academic Proficiency (CAAP)

– Collegiate Learning Assessment (CLA)

The Collegiate Learning Assessment

• Developed by the Council for Aid to Education (CAE) in 2004 – www.cae.org/cla

• A standardized and nationally normed test that measures the institutional contributions (value added) to the learning gains made by students

• Direct assessment of student learning

The CLA measures:

• critical thinking• analytical reasoning / problem solving• written communication

• Time required:– 90 minutes for a performance task (complex

problem to be solved)– 75 minutes for two writing prompts (make an

argument and break an argument)

Student sample:

• First-year students and then seniors• Longitudinal study is better• Cross-sectional study is possible (but more

expensive)• The institution is the primary unit of analysis

– Institutions can be compared for value-added gains

Concerns about the CLA for UC assessment purposes:

• Questionable reliability of results• Lack of integration into the curriculum• Disconnect between the CLA and UC’s mission• Absence of ownership by the faculty• Questionable motivation by seniors taking the

test• High costs to administer the test (over $100

per student)

Recent analyses:

• Trudy Banta, “The Search for a Perfect Test Continues,” in Assessment Update (November-December 2007)

• Joan Hawthorne, “Accountability & Comparability: What’s Wrong with the VSA Approach?” in Liberal Education (Spring 2008)

• Paul Basken, “Electronic Portfolios May Answer Calls for More Accountability,” in The Chronicle of Higher Education (April 17, 2008)

Compare the CLA to e-portfolios:

• Target sample: first-year students, seniors, mid-collegiate students, and Honors students

• Students in the study will take the CLA and also develop e-portfolios.

• The pilot will gather feedback about student perceptions of the CLA and e-portfolios, including perceptions about cost for incentives.

Possible tools: AAC&U Metarubics for

• Integrative Learning• Critical Thinking• Written Communication

More specific research questions:

• How closely does test match institutional goals?

• How motivated to do well are students?• What incentives motivate students?

• Above questions from Trudy W. Banta, “Assessment for Accountability: A Cautionary Tale,” presented at The Ohio State University on June 19, 2008

More possible research questions from Banta’s presentation:

• Which majors perform best?• Is there a ceiling effect?• How valid/reliable are scores and value

added?• How can faculty USE scores to guide

improvement?

UC research time frames:• September-October 2008 for 120 – 150 Honors

students to take the CLA and also begin developing e-portfolios

• The 2008 Assessment Institute in Indianapolis• April/May 2009 for site visit re HLC/NCA

reaccreditation• Four-year time frame within the University System of

Ohio before state-wide adoption of one of the standardized tests• http://universitysystem.ohio.gov/

Another research opportunity:

• “Measuring Critical Thinking and Written Communication to Transition Student Scholars” (UC’s recent proposal for the 2008-09 Paul P. Fidler Research Grant)

And a wild card from Milton Hakel (Ohio Board of Regents Eminent Scholar in Industrial and Organizational Psychology)

From the description of Hakel’s presentation “Accountability – No College Left Behind?” at the Ohio Learning Network’s E-Portfolio Day (July 29, 2008):

• Grades, credit hours, and degrees are no longer credible measures of student learning and success. Standardized achievement testing won’t work any better. That leaves electronic portfolios – bring ‘em on!

Questions? Suggestions?