open badges mooc session 12: james willis

Open Digital Badges

Design Principles Documentation Project

Open edX and Beyond Project

James E. Willis, III, Ph.D.Research Associate

30 organizations awarded grants to develop badge

content in the 2012 Badges for Lifelong Learning

Initiative

Collaborators and Partners

Rebecca Itow

Daniel Hickey

Katerina Schenke

Cathy Tran

Nate Otto

Christine Chow

K-12

Middle School

Secondary

College Stu-

dents

Educa-tors/ Other

Vocational/ Adult

Common Core

Local StandardsPartner

s for 21C

S

None Identified

School or University

After School

Teacher PDIn-for-

mal & Other

Adult & Career

Mu-seum

Extra-Curricu-

lar

Accredited Con-text

Accredited Badges

None

Other

Earners

Setting

Standards

Accreditation

Regardless of where you start, it is likely you will end up somewhere other than your intended destination. That’s okay. Systems are living things and your system needs to be flexible. You need to embrace a bit of chaos in its design.

--Carla Casilli

Most of the knowledge generated when designing complex systems evaporates as features evolve and teams dissolve.

--Phillipe Kruchten

Badge Functions

Recognizing LearningSkills, achievements, experiences, & practicesIndividual, peer, social

Assessing LearningSummative, formative, transformative, & transcendent

Motivating LearningIntrinsic, extrinsic, & participatory

Studying LearningResearch of, for, & with digital badges

Intended Practices- Ideas outlined in original proposals

Enacted Practices- Intentions unfolding in world

Formal Practices- Practices after funding ends

General Findings Across Project

General Findings

Digital Badges are Different

Claims and Evidence are Hard to Define

Many projects struggled to specify claims

Many projects struggled to specify evidence

{

"uid": "f2c20",

"recipient": {

"type": "email",

"hashed": true,

"salt": "deadsea",

"identity":

"sha256$c7ef86405ba71b85acd8e2e95166c4b111448

089f2e1599f42fe1bba46e865c5"

},

"image": "

https://example.org/beths-robot-badge.png",

"evidence": "

https://example.org/beths-robot-work.html",

"issuedOn": 1359217910,

"badge": "https://example.org/robotics-

badge.json",

"verify": {

"type": "hosted",

"url": "https://example.org/beths

-robotics-badge.json"

}

}

General FindingsOpen Digital Badges are Really Different

Info Circulates in Social Networks

Projects recognized that states are increased

Validity gets crowd-sourced

COPPA, FERPA, and other concerns worried many projects

General FindingsIt’s Not (Just) About Badges

Badges are Part of an Ecosystem

More successful projects integrated badge functions into their learning ecosystem

Less successful projects tried to build an ecosystem around badges

SPECIFIC FINDINGS Across Project

Badges Can Really Work

- but not everywhere.

4 Projects Suspended

1 Project Still Trying

7 Projects Partial

18 Projects Implemented

Sheryl Grant’s Starting PointsBadge Systems = Tech + Content + Badges

Bucket Start With Need to Build

Layered Tech + Content Badges

Integrated Tech Badges + Content

Responsive Content Badges + Tech

New Build Badges + Tech + Content

Badges First Badges + Tech + Content

Layered Integrated Responsive New Build Badges First

0

1

2

3

4

5

6

7

8

9

10

11

12

Implementation Status by Bucket

ImplementedPartialPursuingSuspended

K-12

Middle School

High School

Post-Se

con...

Adult0

2

4

6

8

10

12

14Implementation Status by Earners

ImplementedPartialPursuingSuspended

Layered Integrated Responsive New Build Badges First

0123456789

101112

Badge Status by Bucket

Proposed

Different

None

Layered Integrated Responsive New Build Badges First0

1

2

3

4

5

6

7

8

9

10

11

Ecosystem Status by Bucket

Proposed

Different

None

Discussion of One Segment of the DPD Project: Adult Learners

Proposed ecosystem with scaled back practices

Proposed ecosystem with fewer practices

Different badges and ecosystem

Proposed badges and different ecosystem

Still building proposed badges

Proposed badges

Practices Formalized or Proposed & Not Enacted

Category Principle Practice

Recognizing Recognize diverse learning Recognize diverse learning

Use badges to externally communicate learning

Use badges to externally communicate learning

Promote discovery Discover learning opportunities

Assessing Align activities to standards` Internal standards

Use formative functions of assessment

Peer assessment

Use mastery learning Judged by human experts and computer

Use rubrics Rubrics developed for specific artifacts

Promote “hard” and “soft” skills Combine collaborative learning and discreet skills

Motivating Provide privileges Peer mentorship

Display badges to the public Learners can choose to share badges with public

Build outside value for badges Real-life application of knowledge

Proposed & Not Enacted or Unproposed & Formalized

Category Principles General PracticeRecognizing Have experts issue badges Credentialed via accredited

entity and community

Have experts issue badges Credentialed via community

Seek external backing of credential

Externally endorsed

Assessing Use rubrics Rubrics developed for specific artifacts

Motivating Provide privileges Provide prizes

Use different types of assessments

Peer Assessment

Stimulate competition Use point system

Study Study impact of badges Research of badges

Practices Formalized, Proposed & Not Enacted or Unproposed & Introduced

Category Principles General Practice Specific Practice

Recognizing Learning

Use badges as a means of external communication of learning

Endorsement and translation of skills

Promote discovery Discover learners Promote earners’ skills to employers

Assessing Learning

Use leveled badge systems Competency levels Translation of skills

Motivating Learning

Build outside value for badges

Evidence of outside opportunities

Potential to gain employment

Recognize identities Target a specific group Veterans

Engage with community Involvement in digital community

Network with community members

Studying Learning

Study Badge impact Research of badges Data collection and analysis

Practices Proposed & Not Enacted

Category Principles General Practice

Recognizing Use badges to externally communicate learning

Use badges to externally communicate learning

Assessing Use leveled badge systems Competency levels

Enhance validity with expert judgment

Use human experts

Use formative functions of assessment

Provide peer and expert feedback

Involve students at a granular level Learning pathways and badge design

Motivating Set goals User created badges

Use different types of assessments Peer assessment

Principles and Practices Across Projects

Most Challenging RECOGNITION Principles & Practices

Principle Practice Failures/ Attempts

Seek external backing Externally endorsed 6/10Externally valued 6/8

Use badges to externally communicate learning

Use badges to externally communicate learning

4/12

Have experts issue badges Have badges accredited by accredited entity and community

3/10

Align badges to standards Align to internal standards 2/6

Align to national or international standards

2/12

Align to community standards

1/4

Most Challenging ASSESSMENT Principles & Practices

Principle Practice Failures/ Attempts

Enhance validity with expert judgment

Use human experts and computers

4/10

Align assessments to standards and create objectives

Common Core State Standards

3/8

National/state standards 1/8Internal standards 1/9

Use e-portfolios Foster discussion around artifacts

2/3

Open to the public 1/2Local to community 1/4

Use formative functions of assessment

Peer feedback 2/4

Most Challenging MOTIVATION Principles & Practices

Principle Practice Failures/ Attempts

Provide privileges Provide internships 4/6

Provide prizes 3/6

Provide peer mentorship 3/9

Provide new activities 3/11

Use different types of assessment

Peer assessment 4/7

Self assessment 1/3

Expert assessment 1/3

Build outside value for badges

Evidence for outside opportunities

3/9

More Info on the DPD Project: DPDProject.info

Open edX and Beyond Project

“Practical Numerical Methods with Python” MOOC taught by Prof. Lorena Barba,

GWU

Graduate students in on-campus course are simultaneously enrolled

CRLT: technology (facilitating coding in Open edX) and pedagogy (purposeful implementation, evidence, and assessment)

Practical Numerical Methods with Python

Dr. Barba realized that Open edX requires authentication; she proposed a solution: to link directly to GitHub.

May be the first time digital badges have linked to GitHub for evidence.

Challenges and OpportunitiesTechnical

Finding the ‘seams’ in Open edX coding to build a badges API connection

Assuring individual identity verification and management

Keeping open materials within the evidence of outcomes

Pedagogical

Assessing student progress in specific, cumulative skills learned

Aligning outcomes for replication in future edX and Open edX MOOCs

Collaborations to Overcome Challenges

On-ground expertise and collaboration is vital for short-term solution, and to set the stage for the long-term issuance of badges.

Will locate the ‘seams’ in Open edX platform

Commitment to open badges development

Will use API to link to Open edX

Python MOOC: Goals

Short-term goal: by mid-November, a series of badges will be available.

Long-term goal: by spring 2015, instructors in Open edX and edX will be able to seamlessly offer badges.

Present collaboration at Open edX conference in Cambridge, MA in mid-November.

On-Going and Future Goals

Facilitate widespread use of digital badges in higher education

• From fully online courses to hybrid courses and stand-alone open modules.

• From a single platform (edX) to the other major platforms.• From unaccredited MOOCs to for-credit courses, co-

curricular learning, faculty & staff learning.• From email-based & informally verified badges to

cryptographically-signed and formally-verified badges.• From credentialing of isolated learning experiences to

sophisticated dossiers with dramatic impact on college admission and employment.

Publish findings, challenges, collaboration notes, and future opportunities

Questions?