MOOQ for the quality of MOOCs: “We will make MOOCs better” Quality Reference Framework with indicators for design & comparison www.MOOC-quality.eu

Frameworks: MOOQ

Quality of Open Education and MOOCs: Interactive Workshop

@ EARLI 2017 in Tampere, Finland by Christian M. Stracke

Open University of the Netherlands

Open CC License for sharing & re-using slides

This work is free to share under the creative commons licence:

"Attribution – Noncommercial – Share Alike 3.0"

You can copy, distribute and transmit the work under the following conditions:

1. Attribution – 2. Noncommercial –

3. Share Alike

Licence: Attribution-Noncommercial-Share Alike Some rights reserved, see: http://creativecommons.org/licenses/by-nc-sa/3.0/

Hands-on-Session

Explore the quality indicators of MOOCs

MOOQ for the quality of MOOCs: “We will make MOOCs better” Quality Reference Framework with indicators for design & comparison www.MOOC-quality.eu

Frameworks: MOOQ

Our proposal for discussion:

The Quality Reference

Framework (QRF)

Quality Reference Framework with indicators for design & comparison

Our main goal is the collaboration with all to improve future MOOC design for better MOOC learning experiences

16 questions in 4 clusters (n=45): 1. Experience with MOOCs (5 questions) 2. Perception and Expectation (5 questions) 3. Personal Goals (4 questions) 4. Recognition and Certification (2 questions)

n=45: reveals that many MOOC learners do not share the intentions of the MOOC designer & have their own goals

MOOQ Pre-Survey

Three main surveys for learners, designers and facilitators of MOOCs

Please support us to improve next MOOCs: www.survey.MOOC-quality.eu

Your feedback is most welcome!

Global MOOC Survey

Combines 3 surveys on 13 constructs: 1. MOOC learners (69 questions) 2. MOOC designers (89 questions) 3. MOOC facilitators (58 questions)

3 months with huge support (n=625): reveals that most MOOC learners reported positive learning experiences with MOOCs

MOOQ Survey

MOOQ Survey: 1st results

What is Open Education (OE)? Quality dimensions of OE in practice? Quality indicators for OE in practice?

Quality Open Education

1. Pedagogical

2. Technological

3. Business Model

MOOQ Matrix

1. MOOC Learners

2. MOOC Designers

3. MOOC Facilitators

4. MOOC Providers

MOOQ Target Groups

OE Quality Dimensions

Evaluation

Analysis

Implemen- tation

Learning process

Design

Learner

QRF Target Groups

Meso

Micro

Macro

MOOC providers MOOC designers

MOOC facilitators MOOC learners

National ministries Local authorities

2 Parts of the interactive

MOOQ Workshop:

Survey and Quality indicators

Part 1 of the interactive

MOOQ Workshop:

Survey

How much do you agree with the five phases? (Analysis, Design, Implementation, Learning Process, Evaluation)

P-Q1: 5 Phases

Fully

disagree:

Disagree:

Neutral:

Agree:

Fully

agree:

N/A:

0 1 3 11 1 0

The Quality Reference Framework (QRF)

Target groups

How much do you agree with the planned target groups? (National ministries, local authorities, MOOC providers, MOOC designers, MOOC facilitators, MOOC learners)

T-Q1: Target Groups

Fully

disagree:

Disagree:

Neutral:

Agree:

Fully

agree:

N/A:

0 0 1 12 1 0

The Quality Reference Framework (QRF)

Instruments

How much do you agree that the planned instruments are helpful? (MOOC, Guidelines, Evaluation tools, Online communities, Design patterns, Checklists, Use cases, Authoring system, OER)

I-Q1: 9 Instruments

Fully

disagree:

Disagree:

Neutral:

Agree:

Fully

agree:

N/A:

0 0 1 9 2 1

Part 2 of the interactive

MOOQ Workshop:

Quality indicators

OE Quality Dimensions

Evaluation

Analysis

Implemen- tation

Learning process

Design

Learner

Quality dimensions of OE in practice Dimension 1: Analysis Dimension 2: Design Dimension 3: Implementation Dimension 4: Learning Process Dimension 5: Evaluation & Optimization

OE Quality Dimensions

Quality indicators for OE in practice For Dimension 1: For Dimension 2: For Dimension n:

OE Quality Indicators

Quality indicators for analysis in OE: • Content structure • Learning tasks • Characteristics of target groups • Budgets and resources • Learning context • Why provided as MOOC? And not other type? • Convince stakeholders

OE D1: Analysis

Quality indicators for design in OE: • Usability • Accessibility • Constructive alignment of teaching (content

and objectives are aligned) • Interactivity design • Action design • Multimedia design • Motivation design

OE D2: Design

Quality indicators for design in OE: • Quality of learning tasks • Teaching strategy (deductive vs inductive) • Integration of content into narration • Flexibility • Heterogenuous learning paths • Focus on precise definition of learning content

+ what you can learn (goals, structure, intention) • What is the added value of the MOOC?

OE D2: Design

Quality indicators for design in OE: • Level of the MOOC (basics vs expertise) • Strategic plan for realization + marketing • Target audience definition + characteristics • Basic and special features (also for marketing) • Definition of indicators for learning success

(maybe clustering individual intentions into groups of learners)

OE D2: Design

Quality indicators for design in OE: • Question: How are the indicators related to

each other? (dependencies/intertwinned)

OE D2: Design

Q. indicators for implementation in OE: • Platform selection based on criteria for

learning goals and design • Communicate with the stakeholders • Meet the technical conditions + requirements • Create the course area with all assignments

according to the design • Define implementation period • Marketing

OE D3: Implementation

Q. indicators for implementation in OE: • Changing from implementation to preparation?

OE D3: Implementation

Quality indicators for learning in OE: • Collect enrollments before the start of the

Learning process • Scaffolding or help • Enough interactions among peers • Feedback from instructors • Measuring of indicators for learning success

(maybe clustering individual intentions into groups of learners)

OE D4: Learning Process

Quality indicators for learning in OE: • Monitoring system of learning process (that

learners know where they are) • Self-directed feedback through assignments

and self-assessment (in MOOCs without grades) • Assessment of students learning within the

learning process

OE D4: Learning Process

Quality indicators for evaluation in OE: • Kirkpatrick‘s four levels • All quality indicators should be part of the

evaluation (quality criteria for the design have to be evaluated)

OE D5: Evaluation