educational design research: appropriate to address problems in educational practice
DESCRIPTION
Educational Design Research: appropriate to address problems in educational practice. Tjeerd Plomp Professor Emeritus University of Twente Enschede, The Netherlands. - PowerPoint PPT PresentationTRANSCRIPT
Educational Design Research:
appropriate to address problems
in educational practice
Tjeerd PlompProfessor Emeritus
University of TwenteEnschede, The Netherlands
1140519 ICREAMS
Van den Akker (1999; director of Netherlands Institute for Curriculum Development) about research in domain of curriculum development and implementation:many ‘traditional’ research approaches such as experiments, surveys, correlational analyses, with their emphasis on description hardly provide prescriptions that are useful for design and development problems in education
2140519 ICREAMS
In this presentation:
• Research functions and research designs• What is design research• Research question • Quality criteria for interventions• Generalizability in design research• New book on design research
3140519 ICREAMS
Focus of research project
Dependent on research question/goalresearch may address a need
– to describe– to explain, understand, predict– to compare– to evaluate– to design/develop
Called: research functions4140519 ICREAMS
Research approaches/designsto realize research functions e.g.:
• Survey: to describe, compare, evaluate• Case studies: to describe, to explain• Experiments: to explain, to compare• Ethnography: to describe, to understand• Correlational research: e.g. to describe, to
explain• Evaluation research: e.g. to determine the
effectiveness of a program5140519 ICREAMS
Research designs, e.g.:
Design research: to design/develop an intervention (e.g. program, T/L strategy, process, product, system, etc) with the purpose
– to improve practice and contribute to body of knowledge
OR– to develop a theory(and possibly: to inform decision making and policy
development)Note: intervention is container term 6140519 ICREAMS
What is Educational Design Research (EDR)?
Distinction between:
• Development studies aimed at research-based solutions for complex problems in educational practice
• Validation studies aimed at development or validation of a theory
Research on interventions versus research through interventions (McKenney & Reeves, 2012)
Often both angles in one study! 7140519 ICREAMS
What is Educational Design Research (EDR)?
Educational design research – type development studies - isthe systematic analysis, design, development and evaluation of educational interventions with the dual aim of
• generating research-based solutions for complex problems in educational practice, and
• advancing our knowledge about the characteristics of these interventions and the process of designing and developing them.
8140519 ICREAMS
What is Educational Design Research (EDR)?
Educational design research – type validation studies - isthe study of educational interventions – such as learning processes, learning environments and the like – with the purpose to develop and validate theories about such processes and how these can be designed.
9140519 ICREAMS
Educational Design Research encompasses systematic educational design processes.
The reverse is not true:Not all systematic educational design can
be called research
10140519 ICREAMS
11
Iterations of systematic design cycles
analysisdesign & develop
prototype
evaluationRevision: yes?
No? STOP
PROBLEM
Design research methodology can be expressed as follows:
12
ID of problems with practitioners
Development of prototype solutions
Testing solutionsin context
Reflection to producedesign principles
(adapted from Reeves, UoGeorgia, Athens, USA)
Revision
140519 ICREAMS
Susan McKenney, Univ of Twente:
13140519 ICREAMS
Phases in Design Research• Preliminary phase:
needs and content analysis, review of state-of-art of literature => conceptualisation & design
specifications• Development or Prototyping phase
(iterative design phase + formative evaluation) <=> micro-cycles of research
• Assessment phase (semi-summative evaluation)
ALL Phases: systematic reflection and documentation (resulting in
design principles)14140519 ICREAMS
On terminology:
(Educational) Design Researchstands for a ‘family’ of related research approaches
• Design studies, design experiments• Design-based (implementation) research• Development research• Participative action research• Formative research• Engineering research
(e.g. Van den Akker et al., 2006):But a number of characteristics in common!
15140519 ICREAMS
Common characteristics
• Interventionist• Iterative• Process oriented• Utility oriented: merit of design
measured• Theory oriented: based upon &
contributing to• Involvement of practitioners
16140519 ICREAMS
17
In general:Research goal may vary, but always:(i) design/development of intervention (ii) design principles or theory validation
Context Z:
Intervention XInput => Process
OutcomesY1, Y2, …, Yn
A closer look:
140519 ICREAMS
18
Twofold yield in design research
Develop-ment studies
(i) developing a research based intervention as solution to complex problem, and (ii) constructing (re-usable) design principles
Validation studies
(i) designing learning environments with the purpose (ii) to develop and validate theories about learning, learning environments, or to validate design principles
Implemen-tation studies
(i) implementing a particular program and (ii) strategy and conditions under which implementation can happen (design principles).140519 ICREAMS
Research question in design research
If research goal is development of intervention:What are the characteristics of an <intervention X> to realize outcomes Y in context Z
OR in case of developing/validating a theory (tentatively):
What is an adequate learning and teaching strategy for <topic B> in context C in order to cope with the main difficulties in the learning of this topic
19140519 ICREAMS
Examples of research questions
#1 what are the characteristics of an effective
in-service programme for mathematics
teachers through which they develop the
ability to apply student-centred
pedagogical methods (Tecle, 2003)
20140519 ICREAMS
Examples of research questions
#2 What is an adequate learning and teaching strategy for genetics in upper secondary biology education in order to cope with the main difficulties in learning and teaching genetics, and to promote the acquisition of a meaningful and coherent understanding of hereditary phenomena? ( Knippels, 2002)
21140519 ICREAMS
Quality criteria for interventions:1. Relevance or Content Validity:• need for the intervention• based on contemporary scientific insights2. Consistency or Construct Validity3. Practicality
- can be used in settings for which developed4. Effectiveness:
- yields desired results- cost-benefit ratio
#3&4: Expected versus Actual22140519 ICREAMS
Shift in emphasis on criteria
23
Stage Criteria
Preliminary
research
Emphasis mainly on relevance and consistency (i.e. validity),
less on practicality
Development or Prototyping phase
initially: consistency and practicality; increasingly practicality and gradually effectiveness
Assessment phase
practicality and effectiveness140519 ICREAMS
Formative evaluation e.g.: Design
specsGlobal design
Partly developed
Completely developed
Relevance • Screening
• Focus grp
• Screening• Focus group
• Screening• Focus group
• Focus group• Micro-eva
Consistency • Screening
• Focus grp
• Screening
• Focus grp
• Screening• Focus group
• Focus group• Micro-eva
Practica-
lity
• Screening• Focus group
• Screening
• Focus grp
• Walkthrgh
• Focus grp
• Micro-eva
• Walkthrgh
• Focus grp
• Micro-eva
• Try-out
Effective-
ness
• Screening• Focus group
• Screening• Focus group
• Focus grp
• Micro-eva
• Focus grp
• Micro-eva
• Try-out24140519 ICREAMS
25
Context Z:
Intervention XInput => Process
OutcomesY1, Y2, …, Yn
design principles as intervention theorylocal instruction theory
Generalizability in Design Research:Yin (2003) about case study research:striving to generalize findings to some broader theory:
Generalizability in Design Research:Yin (2003) about case study research:
striving to generalize findings to some broader theory
A theory (design principles or local theories) must be tested through replications of findings in a 2nd, 3rd or more cases:
if same results in various cases, then ….the theory (= design principles or local theories) might be
accepted for a much larger number of contexts.
This replication logic is the same that underlies design research!
Analytical generalization.(compare with ‘transferability’ in qualitative research)
26140519 ICREAMS
Edited by Tjeerd Plomp & Nienke Nieveen
Published by: SLO – Netherlands Institute for Curriculum Development. Enschede; 2013
Part A: an IntroductionPart B: Illustrative cases
Free available from:http://international.slo.nl/edr
27140519 ICREAMS
Part B: 51 Cases of Design Research
Case Selection Tool:•Countries •Educational Sectors•Educational domains (>1)•Main aim•Educational focus•Type of intervention (>1)
See: http://international.slo.nl/edr
28140519 ICREAMS
Part B: 51 Cases of Design Research
Case Selection Tool:•22 countries: CHI, SIN, IND, KOR – USA (14), NET (11)•Educational Sectors: all•Educ domains (>1): curriculum (16), learning & instruction (29), subject-related pedagogy (34), school organization (3), instructional technology (7), ICT in education (11)•Main aim: development studies (36); validation studies (14); implemtation studies (1)•Educational focus•Type of intervention (>1)
29140519 ICREAMS
Part B: 51 Cases of Design Research
Case Selection Tool:•Countries •Educational Sectors•Educ domains (>1)•Main aim•Educational focus: teaching-learning methods; curriculum planning; ICT in education; school management & leadership; monitoring quality of educ; professional development; workplace learning•Type of intervention: curr unit/course (34), educ programme (12), learning task (9), monitoring system (4), assessment task (2), other (4)
30140519 ICREAMS
Quotes illustrate that
many researchers in various ‘corners’
of our field belief that there is a need
for an alternative research approach
that is directly relevant for problems in
educational practice.
32140519 ICREAMS
NOTE: One resq => more than one res function
Example:if aim of research isto design and develop a teaching-learning strategy
for acquiring the competency of mathematical modelling (in grade 11 & 12),
then researchers may first want to understand and carefully describe what barriers students experience with mathematical modelling.
But: there is primary research function 33140519 ICREAMS
Why embark on Design Research?
Various reasons – e.g.:
1. Need to solve a complex problem in education practice for which no how-to-do guidelines are available
2. Need to elaborate and/or validate a theory (e.g. whether the theory of Realistic Mathematics Education is valid in a certain context)
34140519 ICREAMS
Problems may exist at all levels - for example:
System level:Need for system for e-learning to serve specific
group of students
Institutional level:What are effective methods for collaborative
learning
35140519 ICREAMS
When is Design Research Appropriate?
Eamonn Kelly (2009):• Initial state(s) unknown or unclear• Goal state(s) unknown or unclear• Operators to move from initial states to goal
states are unknown or how to apply the operators is unclear
36140519 ICREAMS
Generalizability in Design Research:
So: design principles must be seen as ‘heuristic’ statements providing guidance and
direction,but do not give ‘certainties’.
When we give proper weight to local conditions, any generalization is a working hypothesis, not a conclusion!
(Lee Cronbach, 1975, cited by Tom Reeves)
37140519 ICREAMS