educational design research: appropriate to address problems in educational practice

37
Educational Design Research: appropriate to address problems in educational practice Tjeerd Plomp Professor Emeritus University of Twente Enschede, The Netherlands 1 140519 ICREAMS

Upload: cana

Post on 11-Jan-2016

49 views

Category:

Documents


1 download

DESCRIPTION

Educational Design Research: appropriate to address problems in educational practice. Tjeerd Plomp Professor Emeritus University of Twente Enschede, The Netherlands. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Educational Design Research: appropriate to address problems in educational practice

Educational Design Research:

appropriate to address problems

in educational practice

Tjeerd PlompProfessor Emeritus

University of TwenteEnschede, The Netherlands

1140519 ICREAMS

Page 2: Educational Design Research: appropriate to address problems in educational practice

Van den Akker (1999; director of Netherlands Institute for Curriculum Development) about research in domain of curriculum development and implementation:many ‘traditional’ research approaches such as experiments, surveys, correlational analyses, with their emphasis on description hardly provide prescriptions that are useful for design and development problems in education

2140519 ICREAMS

Page 3: Educational Design Research: appropriate to address problems in educational practice

In this presentation:

• Research functions and research designs• What is design research• Research question • Quality criteria for interventions• Generalizability in design research• New book on design research

3140519 ICREAMS

Page 4: Educational Design Research: appropriate to address problems in educational practice

Focus of research project

Dependent on research question/goalresearch may address a need

– to describe– to explain, understand, predict– to compare– to evaluate– to design/develop

Called: research functions4140519 ICREAMS

Page 5: Educational Design Research: appropriate to address problems in educational practice

Research approaches/designsto realize research functions e.g.:

• Survey: to describe, compare, evaluate• Case studies: to describe, to explain• Experiments: to explain, to compare• Ethnography: to describe, to understand• Correlational research: e.g. to describe, to

explain• Evaluation research: e.g. to determine the

effectiveness of a program5140519 ICREAMS

Page 6: Educational Design Research: appropriate to address problems in educational practice

Research designs, e.g.:

Design research: to design/develop an intervention (e.g. program, T/L strategy, process, product, system, etc) with the purpose

– to improve practice and contribute to body of knowledge

OR– to develop a theory(and possibly: to inform decision making and policy

development)Note: intervention is container term 6140519 ICREAMS

Page 7: Educational Design Research: appropriate to address problems in educational practice

What is Educational Design Research (EDR)?

Distinction between:

• Development studies aimed at research-based solutions for complex problems in educational practice

• Validation studies aimed at development or validation of a theory

Research on interventions versus research through interventions (McKenney & Reeves, 2012)

Often both angles in one study! 7140519 ICREAMS

Page 8: Educational Design Research: appropriate to address problems in educational practice

What is Educational Design Research (EDR)?

Educational design research – type development studies - isthe systematic analysis, design, development and evaluation of educational interventions with the dual aim of

• generating research-based solutions for complex problems in educational practice, and

• advancing our knowledge about the characteristics of these interventions and the process of designing and developing them.

8140519 ICREAMS

Page 9: Educational Design Research: appropriate to address problems in educational practice

What is Educational Design Research (EDR)?

Educational design research – type validation studies - isthe study of educational interventions – such as learning processes, learning environments and the like – with the purpose to develop and validate theories about such processes and how these can be designed.

9140519 ICREAMS

Page 10: Educational Design Research: appropriate to address problems in educational practice

Educational Design Research encompasses systematic educational design processes.

The reverse is not true:Not all systematic educational design can

be called research

10140519 ICREAMS

Page 11: Educational Design Research: appropriate to address problems in educational practice

11

Iterations of systematic design cycles

analysisdesign & develop

prototype

evaluationRevision: yes?

No? STOP

PROBLEM

Page 12: Educational Design Research: appropriate to address problems in educational practice

Design research methodology can be expressed as follows:

12

ID of problems with practitioners

Development of prototype solutions

Testing solutionsin context

Reflection to producedesign principles

(adapted from Reeves, UoGeorgia, Athens, USA)

Revision

140519 ICREAMS

Page 13: Educational Design Research: appropriate to address problems in educational practice

Susan McKenney, Univ of Twente:

13140519 ICREAMS

Page 14: Educational Design Research: appropriate to address problems in educational practice

Phases in Design Research• Preliminary phase:

needs and content analysis, review of state-of-art of literature => conceptualisation & design

specifications• Development or Prototyping phase

(iterative design phase + formative evaluation) <=> micro-cycles of research

• Assessment phase (semi-summative evaluation)

ALL Phases: systematic reflection and documentation (resulting in

design principles)14140519 ICREAMS

Page 15: Educational Design Research: appropriate to address problems in educational practice

On terminology:

(Educational) Design Researchstands for a ‘family’ of related research approaches

• Design studies, design experiments• Design-based (implementation) research• Development research• Participative action research• Formative research• Engineering research

(e.g. Van den Akker et al., 2006):But a number of characteristics in common!

15140519 ICREAMS

Page 16: Educational Design Research: appropriate to address problems in educational practice

Common characteristics

• Interventionist• Iterative• Process oriented• Utility oriented: merit of design

measured• Theory oriented: based upon &

contributing to• Involvement of practitioners

16140519 ICREAMS

Page 17: Educational Design Research: appropriate to address problems in educational practice

17

In general:Research goal may vary, but always:(i) design/development of intervention (ii) design principles or theory validation

Context Z:

Intervention XInput => Process

OutcomesY1, Y2, …, Yn

A closer look:

140519 ICREAMS

Page 18: Educational Design Research: appropriate to address problems in educational practice

18

Twofold yield in design research

Develop-ment studies

(i) developing a research based intervention as solution to complex problem, and (ii) constructing (re-usable) design principles

Validation studies

(i) designing learning environments with the purpose (ii) to develop and validate theories about learning, learning environments, or to validate design principles

Implemen-tation studies

(i) implementing a particular program and (ii) strategy and conditions under which implementation can happen (design principles).140519 ICREAMS

Page 19: Educational Design Research: appropriate to address problems in educational practice

Research question in design research

If research goal is development of intervention:What are the characteristics of an <intervention X> to realize outcomes Y in context Z

OR in case of developing/validating a theory (tentatively):

What is an adequate learning and teaching strategy for <topic B> in context C in order to cope with the main difficulties in the learning of this topic

19140519 ICREAMS

Page 20: Educational Design Research: appropriate to address problems in educational practice

Examples of research questions

#1 what are the characteristics of an effective

in-service programme for mathematics

teachers through which they develop the

ability to apply student-centred

pedagogical methods (Tecle, 2003)

20140519 ICREAMS

Page 21: Educational Design Research: appropriate to address problems in educational practice

Examples of research questions

#2 What is an adequate learning and teaching strategy for genetics in upper secondary biology education in order to cope with the main difficulties in learning and teaching genetics, and to promote the acquisition of a meaningful and coherent understanding of hereditary phenomena? ( Knippels, 2002)

21140519 ICREAMS

Page 22: Educational Design Research: appropriate to address problems in educational practice

Quality criteria for interventions:1. Relevance or Content Validity:• need for the intervention• based on contemporary scientific insights2. Consistency or Construct Validity3. Practicality

- can be used in settings for which developed4. Effectiveness:

- yields desired results- cost-benefit ratio

#3&4: Expected versus Actual22140519 ICREAMS

Page 23: Educational Design Research: appropriate to address problems in educational practice

Shift in emphasis on criteria

23

Stage Criteria

Preliminary

research

Emphasis mainly on relevance and consistency (i.e. validity),

less on practicality

Development or Prototyping phase

initially: consistency and practicality; increasingly practicality and gradually effectiveness

Assessment phase

practicality and effectiveness140519 ICREAMS

Page 24: Educational Design Research: appropriate to address problems in educational practice

Formative evaluation e.g.: Design

specsGlobal design

Partly developed

Completely developed

Relevance • Screening

• Focus grp

• Screening• Focus group

• Screening• Focus group

• Focus group• Micro-eva

Consistency • Screening

• Focus grp

• Screening

• Focus grp

• Screening• Focus group

• Focus group• Micro-eva

Practica-

lity

• Screening• Focus group

• Screening

• Focus grp

• Walkthrgh

• Focus grp

• Micro-eva

• Walkthrgh

• Focus grp

• Micro-eva

• Try-out

Effective-

ness

• Screening• Focus group

• Screening• Focus group

• Focus grp

• Micro-eva

• Focus grp

• Micro-eva

• Try-out24140519 ICREAMS

Page 25: Educational Design Research: appropriate to address problems in educational practice

25

Context Z:

Intervention XInput => Process

OutcomesY1, Y2, …, Yn

design principles as intervention theorylocal instruction theory

Generalizability in Design Research:Yin (2003) about case study research:striving to generalize findings to some broader theory:

Page 26: Educational Design Research: appropriate to address problems in educational practice

Generalizability in Design Research:Yin (2003) about case study research:

striving to generalize findings to some broader theory

A theory (design principles or local theories) must be tested through replications of findings in a 2nd, 3rd or more cases:

if same results in various cases, then ….the theory (= design principles or local theories) might be

accepted for a much larger number of contexts.

This replication logic is the same that underlies design research!

Analytical generalization.(compare with ‘transferability’ in qualitative research)

26140519 ICREAMS

Page 27: Educational Design Research: appropriate to address problems in educational practice

Edited by Tjeerd Plomp & Nienke Nieveen

Published by: SLO – Netherlands Institute for Curriculum Development. Enschede; 2013

Part A: an IntroductionPart B: Illustrative cases

Free available from:http://international.slo.nl/edr

27140519 ICREAMS

Page 28: Educational Design Research: appropriate to address problems in educational practice

Part B: 51 Cases of Design Research

Case Selection Tool:•Countries •Educational Sectors•Educational domains (>1)•Main aim•Educational focus•Type of intervention (>1)

See: http://international.slo.nl/edr

28140519 ICREAMS

Page 29: Educational Design Research: appropriate to address problems in educational practice

Part B: 51 Cases of Design Research

Case Selection Tool:•22 countries: CHI, SIN, IND, KOR – USA (14), NET (11)•Educational Sectors: all•Educ domains (>1): curriculum (16), learning & instruction (29), subject-related pedagogy (34), school organization (3), instructional technology (7), ICT in education (11)•Main aim: development studies (36); validation studies (14); implemtation studies (1)•Educational focus•Type of intervention (>1)

29140519 ICREAMS

Page 30: Educational Design Research: appropriate to address problems in educational practice

Part B: 51 Cases of Design Research

Case Selection Tool:•Countries •Educational Sectors•Educ domains (>1)•Main aim•Educational focus: teaching-learning methods; curriculum planning; ICT in education; school management & leadership; monitoring quality of educ; professional development; workplace learning•Type of intervention: curr unit/course (34), educ programme (12), learning task (9), monitoring system (4), assessment task (2), other (4)

30140519 ICREAMS

Page 31: Educational Design Research: appropriate to address problems in educational practice

Thank you!

([email protected])

31140519 ICREAMS

Page 32: Educational Design Research: appropriate to address problems in educational practice

Quotes illustrate that

many researchers in various ‘corners’

of our field belief that there is a need

for an alternative research approach

that is directly relevant for problems in

educational practice.

32140519 ICREAMS

Page 33: Educational Design Research: appropriate to address problems in educational practice

NOTE: One resq => more than one res function

Example:if aim of research isto design and develop a teaching-learning strategy

for acquiring the competency of mathematical modelling (in grade 11 & 12),

then researchers may first want to understand and carefully describe what barriers students experience with mathematical modelling.

But: there is primary research function 33140519 ICREAMS

Page 34: Educational Design Research: appropriate to address problems in educational practice

Why embark on Design Research?

Various reasons – e.g.:

1. Need to solve a complex problem in education practice for which no how-to-do guidelines are available

2. Need to elaborate and/or validate a theory (e.g. whether the theory of Realistic Mathematics Education is valid in a certain context)

34140519 ICREAMS

Page 35: Educational Design Research: appropriate to address problems in educational practice

Problems may exist at all levels - for example:

System level:Need for system for e-learning to serve specific

group of students

Institutional level:What are effective methods for collaborative

learning

35140519 ICREAMS

Page 36: Educational Design Research: appropriate to address problems in educational practice

When is Design Research Appropriate?

Eamonn Kelly (2009):• Initial state(s) unknown or unclear• Goal state(s) unknown or unclear• Operators to move from initial states to goal

states are unknown or how to apply the operators is unclear

36140519 ICREAMS

Page 37: Educational Design Research: appropriate to address problems in educational practice

Generalizability in Design Research:

So: design principles must be seen as ‘heuristic’ statements providing guidance and

direction,but do not give ‘certainties’.

When we give proper weight to local conditions, any generalization is a working hypothesis, not a conclusion!

(Lee Cronbach, 1975, cited by Tom Reeves)

37140519 ICREAMS