using open badges as a certification solution for evaluators
TRANSCRIPT
Using Open Badges as a Certification Solution for Evaluators
Randall S. DaviesDaniel L. RandallRichard E. West
BYUImage by photosteve101. Used under CC BY License.
The Issue of Evaluator Certification
In 1999 the topic of certification was fervently debated
Not everyone feels the AEA needs an evaluator certification
Others feel a certification process would be valuable, and that it may become required for practitioners working with government organizations (Smith et al., 2011)
AEA is no closer to having an evaluator certification than we were a decade ago. Many seem to have concluded that an evaluator certification system may be overly complicated and controversial (Morris, 2011)
AEA
Challenges to Certification
Credentials, Certifications, and Licenses
An Evaluator Certification would evaluate an individual’s Knowledge, Skills, and Experience, verifying that he or she has attained a certain level of expertise.
Any solution AEA might adopt must be– Transparent (Clear Expectations)– Rigorous BUT Flexible and Efficient– Sustainable
AEA
Working Examples of an Evaluator Credentialing Designation
Canadian Evaluation Society’s (CES)
Credentialed Evaluator (CE) designation
This is a designation is not a certification of evaluation proficiency or a license to practice evaluation.
It simply verifies “the holder has provided evidence of the education and experience required by the CES to be a competent evaluator” (http://www.evaluationcanada.ca, np).
AEA
The CE designation is voluntary.
Candidates must have at least two years of evaluation-related work experience within the last ten years, with letters of reference from clients.
They must pass a test requiring a series of narratives (each 150 words or less) adequately addressing at least 70% of the specified evaluator competencies in each of five domains.
AEA
CES Credentialed Evaluator Designation
CES Credentialed Evaluator Designation, Cont.
Members of a credentialing board consider the application and award the designation.
In addition, credentialed evaluators are required to complete ongoing training (40 hours every three years) in order to maintain their status.
This application and renewal process is implemented though an online system designed to manage applications and candidate accounts, with a fee to cover administration costs.
AEA
Issues & Concerns AEA Must Consider
Training Concerns:– Degrees vs training, Experience without a degree
(capacity building). What combination of training, Skill, and Experience is needed.
Assessment Concerns:– Certification Exams vs Credentials, Validity of
assessment.
Administration Issues:– Costs involved in establishing and maintaining a
systemAEA
Potential for an Open Badges Solution
Transparency:– Badges communicate specifics, Meta-data,
Flexibility:– Modification is relatively easy, Skill subsets can be defined, badges
can be added or revised individually.
Rigor:– Individuals Authorized to Award Badges, Allows for levels of
proficiency to be identified (basic – advanced)
Management Sustainability:– Top down design, Automation of much of the process possible,
Division of labor model
AEA
Merit Badges and Digital Badges
Boy Scout Merit Badges
Digital Badge
- Acknowledge accomplishment- Display skills gained- Motivation
Same Benefits as physical badges -Typically not sharable -
Gamification -
Khan Academy Badges (not Open!)
Retrieved from https://www.khanacademy.org/badgeson 11/30/13.
Duolingo (not Open Badges)
Screenshot from the Duolingo app on an Android Tablet. 1/24/14.
Encourages daily practice
Completed badge
Progress indicators
Competition with others is optional
Previously earned badge not at full strength
Open Badges
Open Badges
Same Affordances as Digital Badges, Plus: - Uses Open Badge Infrastructure (OBI)- Display badges via web- Metadata (Criteria and Evidence links)
Open Badge Metadata
Mozilla Backpack
Backpack Collection
Multiple collections can be created.
Collections can remain private or can be made public and shared.
Mozilla’s Vision of Credentialing
Learn and gain skills in formal and informal settings
Collect and Display Badges
New Opportunities
Applications
Gamification: breaking education into achievable levels where progress is recognized and rewarded
Duolingo, Khan Academy, many others
Recognizing expertise within a community Coding communities, translation communities
Marrying informal and formal learningConcordia, WGU
Providing more authentic credentials, tying evidence to criteria to credential
IPT Ed Tec
Strong criteria• A badge’s value is based on the rigor of the criteria for
earning itEasy submission
• Make it simple for people who deserve a badge to get itWeb-based evidence
• Portfolios, links, Google documents, etc. Allows for evidence to be viewed with a badge
Skilled assessments• It is essential to trust the person making the assessment
Open and shareable badge collections• What’s the point if you can’t share your badges in multiple
ways?
Key Principles of Badging
iMovie PersonalTech
TechnologyIntegration
One Possibility for an AEA Badge SystemA E A
Mozilla’s Webmaker badges plan used the “constellation” concept in which smaller
badges are earned and added together to receive a higher level badge.
Badges image adapted from the Mozilla Open Badges Project. Retrieved from http://erinknight.com/post/29830945702/webmaker-badges on 11/9/12.
Early Badge System by Mozilla
Google Sites
PersonalTech
Choice1
Choice2
Choice3
IPT EdTec Badge System
Student selected Internet Communication Technology
Student selected Multimedia Technology
Student selectedPersonal Technologies
AdditionalConcepts
MobileLearn
Internet Safety
Copy right
iMovie
Lower level badges are not issued for these projects
Project level badge not issued for these
additional concepts
Educational Technology
Course Level Badge
Project Level Badge
Lower Level Badges
iMovie PersonalTech
First Iteration
Theory
Skill
Experience
TechnologyIntegration
Certified EvaluatorLevel Badge
2ndLevel Badge
3rdLevel Badges
Possible AEA Badge System
Anatomy of a CE Badge
Pips act as stars:1 = Basic 2 = Intermediate3 = Advanced
Outer ribbon signifies this badge
is the “Certified Evaluator” (CE)
badge
Each badge indicates what category
the badge is from (theory, experience, skill, or
green for the CE badge). Center of badge is reserved for
images to individualize
badges
Pips and the CE Badge
7 - 9 pips earned from 2nd level
Basic
Intermediate
Advanced
4 - 6 pips earned from 2nd level
At least 1 pip from each category
Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is
due: Designing Open Badges for a technology integration course.
TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5
View our presentation at the Open Education Conference by clicking here.
To Learn More…
Randall S. Davies – [email protected]
Daniel L. Randall – [email protected]
Richard E. West – [email protected]
Contact us with Questions
Thank You