case study: education portfolio service...

Case Study: Education Portfolio Service System

Erin Knight, Jin Young Baik, Nathan Gandomi

Table of Contents

A SERVICE SYSTEM REDESIGNED 2

BACKGROUND 2

THE E‐PORTFOLIO 3

EDUCATION PORTFOLIO SERVICE SYSTEM (EPSS) 4

BACKSTAGE INFORMATION FLOW IN THE EPSS 4

FRONTSTAGE – STUDENT EXPERIENCE 5 FROM WITHIN THE SMC 5 FROM WITHIN THE PORTFOLIO SERVICE SYSTEM 6

BENEFITS / CONSTRAINTS 9 EPSS BENEFITS 9 DISCUSSION OF CONSTRAINTS 9

CONCLUSIONS 9

FUTURE WORK 11 IMPLEMENTATION 11 ADDITIONAL CONTEXTS 11


2

A Service System Redesigned For the fall semester 2009 at University of California – Berkeley’s School of Information, our team of graduate students, including Erin Knight, Nathan Gandomi and Jin Young Baik, designed a service system for the final project in Information Service and Service Design. Using the techniques and methodologies learned in the course, the team revisited the traditional e‐Portfolio system and designed a more holistic service system that would enable reflection and synthesis of course material, discovery of connections across courses, and reuse of learning, while limiting the burden on the student and optimizing the organization of the portfolio content. A detailed case study of our redesigned Education Portfolio Service System (ePSS) follows.

Background A typical course experience often includes many documents and resources in multiple formats that students must manage outside of the lecture session itself, including instructor slides, readings, student notes, handouts, assignments, and more. Some progressive classrooms are introducing participatory tools such as wikis or blogs, to encourage students to add to the course content and supplement the experience with their own thoughts and examples. While this helps to create a more flexible environment that adapts to students' needs and interests, it is more content that each student must manage. And this issue increases exponentially when we consider that students are often in multiple courses at once.

Figure 1: The document space for a particular course. Students must manage many different documents, often in different formats. When they attempt to revisit the learning space, to study for an exam for example, the student must pull together the documents and content and try to replicate the experience, which is difficult and unlikely.

Many Learning Management Systems and coursesite solutions attempt to mitigate these issues by offering a single online location to go for the course content. But for the most part, current systems are designed in a way that keeps course content isolated and confined to each particular course instance. This means that content is not available to the student across courses in a collective and accessible way. And these systems


3

actually introduce a new problem in that the content is then "stuck" within the coursesite, and is often gone or inaccessible when the course is completed. While students may keep their set of personal notes or produced content such as papers or assignments, that information, if managed at all, is potentially never reconnected with the actual course content, nor is it connected with other relevant concepts from other courses. Thus, it is not possible to see collective content and learning paths across time. Additionally, there is no way to capture the social learning that occurs through forum discussions, blog posts, and other social media tools within each course and across courses. As a result, at the end of a student’s “lifetime”, no single, cohesive artifact exists that enables a user to reflect upon, synthesize or reuse their learning.

The e‐Portfolio In response to this problem space, a particular type of solution has emerged, the e‐Portfolio, which is defined as “a collection of authentic and diverse evidence, drawn from a larger archive, that represents what a person or organization has learned over time, on which the person or organization has reflected, designed for presentation to one or more audiences for a particular rhetorical purpose.”1 e‐Portfolios have three key components:

• Content Management – storage and management of content, typically student‐generated content

• Lifelong access – access to content after graduation or the end of the student’s affiliation with a particular institution

• Packaging tools – repurposing content for resumes, websites, applications, etc.

A comprehensive competitive analysis of the e‐Portfolio space demonstrated that there are a lot of existing players, which emphasizes the value of the concept and the general need for a way to store education content and repurpose it for other uses. All competitors were built around the three key components mentioned above, but the analysis also demonstrated two other common themes:

• Student‐generated Content Only – The portfolio consists of only the content that the student creates, such as assignments, papers or essays.

• Student‐driven ‐ Students publish, promote or upload all portfolio content to the portfolio system and then organize it themselves.

Thus, existing the e‐Portfolio options put the entire burden on the student to explicitly collect and upload content to the portfolio, and then organize it themselves. This limits the capacity for reflection and synthesis since the student must decide now what they feel is important, with little insight into what they may want to come back to at a later point. Additionally finding connections across resources and courses is constrained by relying on the student to build the portfolio one resource at a time, and organize the portfolio based on their current understanding of the material.

We believe an incredibly valuable service that is missing from the education space is a portfolio service system that automatically collects, stores and maintains course and student‐generated content over time and presents the total learning path content to the student in a meaningful and reusable way.

1 E‐Portfolios. (2005). EDUCAUSE Learning Initiative. http://www.educause.edu/ELI/Archives/EPortfolios/5524


4

Education Portfolio Service System (ePSS) Our new system, the ePSS, involves an extension of the current coursesite environment across courses and across a user's "lifetime" as a student and beyond. This system captures not only student‐generated content, but also the complete learning trail, including course and social content. Additionally that content "capture" does not depend on student submission or promotion only, thus limiting the burden on the student and creating a more comprehensive collection for future reflection and discovery purposes.

The ePSS features:

• Automated content management and organization for all student‐produced content, course content and social content, across courses

• Portfolio views that allow students to transverse each learning trail, re‐experience content sequentially, and discover connections across the total learning graph

• Lifetime access to all content, even beyond graduation • Portfolio package/transformation tools to enable easy customized packaging of content for use

outside of the system (interviews, meetings or applications, for example)

One thing to note, due to the time constraints and scoping necessities, the ePSS is specifically designed around higher education and is tied directly to the Social Media Classroom2 (SMC). The SMC is a coursesite solution that manages all course content and also includes embedded social media tools such as wikis, blogs, forums and social bookmarks to enable students to contribute content as well. The vision is that each course the student is taking would be using the SMC and their single portfolio would be connected to each course.

Backstage Information Flow in the ePSS The ePSS is an information‐intensive service, and much of the information exchange occurs backstage by the system itself. That is by design, to limit the burden on the student and ensure that a rich, reflective set of content is saved within the portfolio.

Content Sync – When a student initiates a Social Media Classroom (SMC) session, the ePSS syncs all new content3 since the previous session, and pulls any new information that is not flagged and is accessible to the students, as well as all associated metadata, into the portfolio.

Content Organization/Connection – Once information has been pulled into the portfolio, the ePSS organizes the content using the metadata. This metadata includes key tags such as the controlled vocabularies specified by the SMC administrator, collective tags added by the students in the course, voting or rating information where applicable, and contextual metadata such as timestamp, revision information, file format, etc. This provides a rich set of data that the ePSS can use to organize and connect the portfolio content.

Each of these steps occurs in the backstage, without any action by or burden on the student. There are a few frontstage elements which provide the user some controls around the information flow (discussed below), namely to exclude information or add metadata, but these actions only further refine the portfolio, they are not required to ensure the portfolio is robust and reflective.

2 Social Media Classroom. (2009) http://socialmediaclassroom.com

3 As previously mentioned, the SMC is a coursesite solution that stores all course content, including syllabi, lecture notes, student notes and social media content.


5

Frontstage – Student Experience To understand the student experience, we will follow Jean, a typical student, through her experience with the ePSS.

From Within the SMC As mentioned above, the ePSS syncs the new content and associated metadata, and organizes it within the portfolio, without any student action needed. Therefore, the student experience with the portfolio from the coursesite is very minimal, if any. Basically, students can add metadata to content for the portfolio, or flag the content to be excluded.

Jean views content within the SMC, in this case, a blog by another student. The only difference from the existing SMC is the Portfolio link at the top near View. When there is an ePSS connected with an SMC instance, this link is available on all accessible content within the site.

Jean can click over to the Portfolio settings to alter the default settings, which are to include the content in the portfolio with the existing metadata. Because another student added this blog, Jean cannot change the associated metadata within the SMC, but she can change the metadata that will be pulled into her portfolio. She can delete the existing tags or add new tags. She can also flag the


6

content to exclude and the ePSS will not pull it into the portfolio, or if is already in the portfolio, will remove it on the next sync.

From Within the Portfolio Service System Most of the student interaction with the system is from within the portfolio itself. The portfolio is accessible directly via the designated URL, or a link in each SMC instance. If Jean accesses the portfolio outside of an SMC instance, she will have to log in, but if she accesses it through an SMC, the system will log her in automatically and she will see the learning trail associated with that course and SMC instance.

There are three key views in the portfolio:

1. Learning Trail – Each course has an associated learning trail, defined by the key tags from the SMC, such as lecture tag (“L1”, “L2”). The student can view the learning trail and drill down into any particular node to view all the content associated with it. This gives the student a visual representation of their learning experience, and can help them easily find and revisit information.

The trail represents all of the content from that particular course, organized by the key tags, in this case, the lecture tags (“L1”, “L2”). The key tags are defined through the SMC by the SMC administrator, as they are now. As shown above, Jean can drill down into any particular node and view all the content that is associated with it – slides, her notes, readings, blog entries, etc. This view gives her a holistic view of the course, and allows her to manage all relevant resources and find them easily.

2. Document Timeline – Another view is the timeline view, which displays content sequentially and allows the student to “flip” through the content. The student can view content from the entire trail, within a particular node, or by a single tag. The document timeline view is especially helpful for revisiting particular topics or studying for exams, because the student can revisit all of the learning in the same order it was experienced in the course or courses.


7

Another view into the same content is the Timeline view. This displays the content in a sequence, as it was chronologically experienced in the course. In this particular screenshot, Jean is viewing all the content associated with Lecture 2 (“L2”), but she could also view all the content tagged with any tag, or the entire trail itself. This view gives her an easy way to browse by topic, or re‐experience the content as she did in the course.

3. Learning Graph – Each course has its own associated learning trail, and the student can “zoom out” to see the broader learning graph with all of the trails. This view allows students to discover connections across courses and easy traverse the entire learning space.

The final view is the learning graph view. Because Jean is taking several courses, she has a trail for each course in her portfolio. She can “zoom out” to see the entire graph which can help her find connections across courses. She can mouse over the connector to see the common tag(s) across courses, and can easily click the tag to view all the relevant content.


8

Other frontstage capabilities from within the ePSS:

Search – Jean can always bypass the views and search to get directly to a particular piece of content.

Reflection – As Jean views content, she can add a note to anything in the portfolio, allowing her to reflect on her learning, record her synthesis and add it to the trail for later use.

Upload of content – Jean can upload content directly to her portfolio, bypassing the SMC. This is useful if one of her courses is not using the SMC, or if she wants to add material from outside of the course context. Additionally, this will be a very important feature once she has graduated, since this will be her main way to add to her portfolio across her lifetime.

Transformation of content – Another key part of lifelong learning and access, is information reuse. Therefore, the ePSS features easy transformation tools that enable Jean to output specific portfolio content into various formats such as PDF, HTML and XML, for other purposes such as public portfolios, resumes or applications.

Jean decides to output the entire course trail, perhaps for a study guide, and the ePSS gathers all the data and outputs it for her. She has additional controls for each format, including a Creative Commons license.


9

Benefits / Constraints

ePSS Benefits • Automatic content capture puts little to no burden on the student

• Comprehensive collection of learning content facilitates deeper reflection and synthesis

• Interconnected content enables cross‐course discovery

• Lifelong access and transformation tools help students reuse this information later in life (job interview, application, etc.)

Discussion of Constraints Information organization / overload – There is the potential for a large amount of data and information influx since it is automatically gathered from each course. The key to the design is the metadata. There is an incredibly rich set of metadata in the SMC, especially due to the social and participatory nature of the site. This data will enable the system to manage the content and organize it in a way that is easy‐to‐use, navigate and discover connections through. Unlike the competitors, who force the student to organize the content, typically within a folder structure, the ePSS offers multiple ways to find and view content.

Tied to SMC (all courses might not be using it) – Initially, the ePSS was conceived as an LMS‐independent system to support all students and be as flexible as possible. The issue with that approach is that the system cannot rely on a consistent and rich set of metadata, which is critical to the organization and management of the content. The Social Media Classroom is a lightweight, open solution that affords many types of content, including social content, so we decided to focus the ePSS design around the SMC. This is a constraint in that not many teachers are using the SMC, but that may change. It can be argued that most teachers recognize the benefit of lifelong learning and learning reuse, so this could actually be a catalyst to get more people to use the SMC. Additionally, the ePSS is really a proof of concept for a revolutionary new kind of e‐Portfolio that automatically captures and organizes content. Eventually, any of the LMS providers could build a similar system from our model, or perhaps our model could become a standard that different systems adopt.

Open and lifelong ‐ financially feasible? – We are committed to open and accessible learning solutions, like the SMC. But the reality is that supporting “lifelong” access means storing and managing a potentially large amount of content for a long time, which could be financially infeasible. We feel that this can be evaluated as adoption increases, but it may be that hosted clients pay an additional fee or that after a certain memory capacity, the student pays for additional space as many competitors do now.

Copyright – In the digital world, copyright lines can be gray, and this is especially true for the ePSS, which is pulling content from courses and saving it for a very extended period of time. Therefore, this system and the connected SMC must be employed under a Creative Commons license so that all content created through the system is covered and can be stored and reused in the portfolio without violating copyright. This would include instructor notes, student notes, projects and social content. The only remaining issue is the readings, most of which are copyrighted, so the ePSS will only pull publicly available reading content into the portfolio, and will just include pointers to the copyrighted material.

Conclusions The ePSS is a new and revolutionary e‐Portfolio system that automatically pulls content from the course site and organizes it within the portfolio, eliminating the student burden and capturing the full scope of


10

educational material. This enables more reflective power in that the student can come back to the content after an extended period of time and re‐experience or reuse the learning. This also supplements the portfolio with more than just the traditional student‐generated content such as papers or essays, and captures the instructor‐provided authoritative information and the social learning content as well.

Remember this? The typical document space for a course environment.

The ePSS streamlines the experience making it easier to revisit content, reflect, synthesize and reuse learning.


11

The ePSS is a proof‐of‐concept of a personal learning environment that can grow around a student’s lifetime as a student and beyond, without over‐burdening or constraining itself to what the student thinks to add at this particular point in time. With this report, we hope to spur some discussions and thoughts in the education space about this new approach to lifelong learning and learning reuse.

Future Work

Implementation We would like to see the ePSS implemented across several SMC instances to dive more deeply into the information model and see it in action. Additionally, it would be great to see the concept adopted and adapted for other LMS systems as well.

Additional Contexts The Portfolio Service System is an information‐intensive service since the key value is derived from the exchange of information, specifically from the SMC to the portfolio, and from the portfolio to the user. The system currently spans several of the contexts from the seven contexts of service design4, and has the potential to include even more:

Both the Person‐to‐Person (P2P) and Technology‐enhanced P2P contexts are really handled through the classroom experience and the SMC. Since the portfolio is tied to the SMC, it bridges these contexts. At its core, the Portfolio Service System is a Self‐Service context system in that each student will have their own portfolio, which they can access, traverse, edit, transform, export or even delete entirely, directly through the portfolio web interface. Additionally, the crux of ePSS is the automatic capture and organization which all occurs in the Computational Backstage context by the system itself. And finally, the ePSS could be considered to fall under the Multi‐channel context as well, since the transformation tools in the portfolio give the student the ability to reuse the information in many different channels – including offline documents, websites and more.

Potential Future contexts:

• Multi‐device – Mobile access to the portfolio and associated learning trails and graph would be of great value, so that the reflective and reuse capacities of the portfolio can be extended to an anytime, anywhere context.

• Context‐aware – Somewhat aligned with the Multi‐device context, if the ePSS had information about the student’s location and other contextual data, it would only make the portfolio metadata even more robust.

4 Glushko, R. (2010). Seven Contexts for Service System Design. Handbook of Service Science, in press.

case study: education portfolio service...

Documents