next practices for oer quality evaluation | lisa petrides
Post on 28-Nov-2014
7.439 Views
Preview:
DESCRIPTION
TRANSCRIPT
Next Practices for OER Quality Evaluation: Using Analytics to Support Continuous Improvement
LAK 2013 - Learning Object Analytics for Collections, Repositories & Federations
April 9, 2013
4.9.13
Lisa Petrides, Ph.D.
ISKME 2013:
ISKME (Institute for the Study of Knowledge Management in Education)Research, tools and services to advance teaching and learning
• Study (research)• Open (open knowledge networks)• Build (training and design)
OER Commons.org
Open Author
Open Key drivers and next practices
Next practices:OER evaluation tools
Custom analytics
00
Uncertainty around implementation of new learning standards
Decreases in education funding
New learning standards
Increased demand for analyticsrelated to the use of online resources
Analytics – For What PurposeFrom resource discovery to improved teaching and learning
• What resource usage patterns can and should be tracked and shared? • How can paradata
support resource and technology improvements?
• Which resources are learners spending time on? • How do usage
patterns map to assessment outcomes?
• Are resources meeting learning standards? If yes, how? If no, why not?• What factors make
resources reusable by teachers/learners? • What makes an
exemplary resource exemplary?
Learning Registry
NSDL schema for paradata exchange
Site-specific initatives*
*Examples include: Open High School Utah, Carnegie Mellon OLI, edX and others
ISKME – OER Commons
• How can we support resource discovery through shared metadata and paradata standards?
Key questions
Resource discovery
Technology and resource improvements
Curriculum improvements
Target outcomes
Enhanced teaching and learning practices; Curriculum improvements
OER Quality EvaluationEQuIP tool for evaluating resources on alignment to state standards
Rubric dimensions:
1.Alignment to the depth of the CCSS (Common Core State Standards)
2.Key shifts in the CCSS
3.Instructional supports
4.Assessment
5.Overal rating for the lesson/unit
OER Quality EvaluationAchieve tool for evaluating resources on quality dimensions
Rubric dimensions:
1.Quality of explanation of the subject matter
2.Utility of the materials designed to support teaching
3.Quality of assessments
4.Quality of technological interactivity
5.Quality of instructional and practice exercises
6.Opportunities for deeper learning
Analytics Use Case Supporting teacher professional development around finding, creating, evaluating and aligning resources
• Are my teachers finding the resources they need?• Are they reaching our district’s goals for identifying
and evaluating resources?• What activities do teachers need more support in?• Where should I focus my professional development
efforts with my teachers?• Are teachers able to see what dimensions of a
resource need to be improved for it to be
considered exemplary?
Project leaders, district administrators, and state curriculum developers working with teachers to identify quality resources that are aligned to learning standards
Key Questions the Analytics Help to Answer
• What distinguishes resources with high ratings from those with low
ratings?• What is it that makes a resource exemplary?• What factors contribute to the use and reuse of resources by
teachers?• How can we encourage the creation of high quality resources
through our tools and supports?
Key Questions the Analytics Help to Answer (for ISKME)
Analytics Use CaseSupporting improvements on learning resources
Example Dashboard ViewResources by evaluation scores
Quality of explanation of subject matter
Quality of technologicalinteractivity
Example Dashboard View Evaluation activities by user
Michael Sander
Jessalyn Katona
Marta Levy
William Donovan
Avery Mitchell
Sam Olsson
Chris Senges
Indicates whether goals for evaluating (or tagging) resources have been met
Example ReportUser comments on evaluated resources
• All qualitative comments can be exported to a csv file for content analysis
• Comments can provide insight into needed improvements to the resource, what is good about the resource, and ways the resource can be used in the classroom
Next Phase Custom AnalyticsExamples of additional data we are collecting through Open Author
Open Author Analytics Indicator of….
# of subheadings by resource Whether resources can be broken into smaller parts. How “modular” is the resource collection?
# of external URLs by resource Whether resources are being combined with other resources. How “remixable” are the resources?
# of versions of a resource by the original author; # of versions by other others
How many derivatives are being made of the resources, and by whom? Are resources in the collection adaptable?
Reasons provided by users for changing an existing resource
Why and how resources are changed.What makes a resource adaptable?
What This All MeansContinuous improvement of resources toward enhanced learning
If one of our hyptheses is correct that…
Resources with the highest overall quality rating on our Achieve rubric are also found to have:
•The highest rating on dimension 5: Quality of instructional and practice exercises•More subheadings than other resources (more modular)•More external URLs than other resources (more remixable)•More versions created (more reusable)
•ISKME builds prompts into Open Author to encourage the creation of resources that have these components•This leads to the creation of new resources that potentially better meet learning standards and teaching needs•The newly created resources are then analyzed through the analytics•This creates a continuous cycle of resource and tool enhancement, towards improved teaching and learning
This could lead to…
Lisa Petrides, PresidentEmail: lisa@iskme.orgTwitter: @lpetrides
Institute for the Study of Knowledge Management in Education
Half Moon Bay, California
top related