http://er.aera.netEducational Researcher

http://edr.sagepub.com/content/32/1/5The online version of this article can be found at:

 DOI: 10.3102/0013189X032001005

2003 32: 5EDUCATIONAL RESEARCHERDesign-Based Research: An Emerging Paradigm for Educational Inquiry

  

 Published on behalf of

  American Educational Research Association

and

http://www.sagepublications.com

can be found at:Educational ResearcherAdditional services and information for    

  http://er.aera.net/alertsEmail Alerts:

 

http://er.aera.net/subscriptionsSubscriptions:  

http://www.aera.net/reprintsReprints:  

http://www.aera.net/permissionsPermissions:  

What is This? 

- Jan 1, 2003Version of Record >>

at UNIV OF NEW HAMPSHIRE on February 22, 2013http://er.aera.netDownloaded from

5JANUARY/FEBRUARY 2003

The authors argue that design-based research, which blends empir-

ical educational research with the theory-driven design of learning

environments, is an important methodology for understanding how,

when, and why educational innovations work in practice. Design-

based researchers’ innovations embody specific theoretical claims

about teaching and learning, and help us understand the relationships

among educational theory, designed artifact, and practice. Design is

central in efforts to foster learning, create usable knowledge, and ad-

vance theories of learning and teaching in complex settings. Design-

based research also may contribute to the growth of human capacity

for subsequent educational reform.

Educational researchers, policymakers, and practitionersagree that educational research is often divorced from theproblems and issues of everyday practice—a split that cre-

ates a need for new research approaches that speak directly to prob-lems of practice (National Research Council [NRC], 2002) andthat lead to the development of “usable knowledge” (Lagemann,2002). Design-based research (Brown, 1992; Collins, 1992) isan emerging paradigm for the study of learning in context throughthe systematic design and study of instructional strategies andtools. We argue that design-based research can help create andextend knowledge about developing, enacting, and sustaining in-novative learning environments.

Definitions of design experiments abound (see Bell, 2002a). Weuse the phrase design-based research methods deliberately (afterHoadley, 2002) to avoid invoking mistaken identification with ex-perimental design, with studies of designers, or with trial teachingmethods. We propose that good design-based research exhibits thefollowing five characteristics: First, the central goals of designinglearning environments and developing theories or “prototheories”of learning are intertwined. Second, development and researchtake place through continuous cycles of design, enactment, analy-sis, and redesign (Cobb, 2001; Collins, 1992). Third, research ondesigns must lead to sharable theories that help communicate rel-evant implications to practitioners and other educational design-ers (cf. Brophy, 2002). Fourth, research must account for howdesigns function in authentic settings. It must not only documentsuccess or failure but also focus on interactions that refine our un-derstanding of the learning issues involved. Fifth, the developmentof such accounts relies on methods that can document and con-nect processes of enactment to outcomes of interest.

Design-Based Research: An Emerging Paradigm forEducational Inquiryby The Design-Based Research Collective

Educational Researcher, Vol. 32, No. 1, pp. 5–8

Why We Need Design-Based Research: Challenges of Context, Design, and Enactment

The last few years have seen a renewed effort to close the “credi-bility gap” (Levin & O’Donnell, 1999) in educational research.Some see this gap as arising from unscientific research approaches(e.g., NRC, 2002), while others point to the detachment of re-search from practice (Lagemann & Shulman, 1999). Educa-tional research that is detached from practice may not accountfor the influence of contexts, the emergent and complex natureof outcomes, and the incompleteness of knowledge about whichfactors are relevant for prediction (Robinson, 1998). Claimingsuccess for an educational intervention is a tricky business. If suc-cess means being certain that an intervention caused learning,then we need to look carefully at the intervention in a particularsetting. However, research in this model would be difficult togeneralize to other settings. On the other hand, if success meansbeing able to claim that an intervention could be effective in anysetting, then we should study effects across a variety of settingsin order to generalize. However, this kind of research leaves manyquestions unanswered about how any observed learning wascaused by interactions between intervention and setting. To ad-dress these problems, we view educational interventions holisti-cally—we see interventions as enacted through the interactionsbetween materials, teachers, and learners. Because the interven-tion as enacted is a product of the context in which it is imple-mented, the intervention is the outcome (or at least an outcome)in an important sense.

In addition, the design of innovations enables us to createlearning conditions that learning theory suggests are productive,but that are not commonly practiced or are not well understood.For example, the Jasper Series (Cognition and Technology Groupat Vanderbilt, 1997) was one of the first learning environmentsthat presented students with an opportunity to develop compu-tational skills by grappling with real-world scenarios. This inno-vation allowed the testing of the tenets of “anchored instruction,”which included the belief that learning should be contextualized,and of ideas that mathematics learning should be more closelytied to students’ experience. Successive classroom trials with dif-ferent versions of these series contributed to an understanding ofthe characteristics of an effective “anchor,” clarified issues for thetheoretical refinement of transfer, and showed how social inter-actions play a role in metacognition.

Illustrations of Design-Based Research

Design-based research methods focus on designing and exploringthe whole range of designed innovations: artifacts as well as less

at UNIV OF NEW HAMPSHIRE on February 22, 2013http://er.aera.netDownloaded from

EDUCATIONAL RESEARCHER6

concrete aspects such as activity structures, institutions, scaffolds,and curricula. Importantly, design-based research goes beyondmerely designing and testing particular interventions. Interven-tions embody specific theoretical claims about teaching andlearning, and reflect a commitment to understanding the rela-tionships among theory, designed artifacts, and practice. At thesame time, research on specific interventions can contribute totheories of learning and teaching.

One example is the BGuILE project (Reiser et al., 2001), aprogram of research to support inquiry learning in biology. Theunderlying innovative approach was the use of discipline-specificscaffolds for data analysis and explanation construction. Initialdesign efforts focused on developing software and activity struc-tures to support student investigations of episodes of natural se-lection, followed by cycles of design, enactment, and analysis tosuccessively refine the relationships between technological scaf-folds for inquiry and social scaffolds to support scientific dis-course. Besides improving the curriculum design, this projectyielded concomitant findings concerning psychological aspectsof students’ understanding of scientific explanation and argu-mentation (Sandoval, 2003), aswell as the role that artifacts andthe social interactions aroundthem play in such processes(Tabak & Reiser, 1997). Suchfindings provide insights intothe complexity involved in de-veloping knowledge and skills,and they help us understandthe role that teachers play incapitalizing on the affordancesof learning materials, but theycould easily have gone unno-ticed had the research focusedsolely on the summative effectsof the intervention.

Design-based research methods respond to emergent featuresof the setting. For example, the Passion School Project (Joseph,2002) articulated theoretical principles that should drive aninterest-based curriculum, including the principle that studentsdevelop competencies in adult-defined learning objectivesthrough engagement in authentic work in their area of interest.Micro-analyses of student interactions with activities based onthat principle enabled redesign and refinement of activities, andultimately refinement of the underlying interest-driven learningframework. Thus, emergent behaviors of students in response toactivities drove development of the intervention and develop-ment of theory. These developments would have been unimag-inable in the absence of real student choices.

Finally, in design-based research, practitioners and research-ers work together to produce meaningful change in contexts ofpractice (e.g., classrooms, after-school programs, teacher on-linecommunities). Such collaboration means that goals and designconstraints are drawn from the local context as well as the re-searcher’s agenda, addressing one concern of many reform efforts(Robinson, 1998). Engaging such partnerships across multiplesettings can uncover relationships between the numerous vari-

ables that come into play in classroom contexts and help refinethe key components of an intervention. In particular, these part-nerships can help us distinguish between a “lethal mutation”(Brown & Campione, 1996)—a reinterpretation that no longercaptures the pedagogical essence of the innovation—from a pro-ductive adaptation—a reinterpretation that preserves this essence,but tailors the activity to the needs and characteristics of partic-ular classrooms. For example, Baumgartner (1999) describedhow several teachers adopted different strategies to manage thetension between performance goals (e.g., building effective fish-ing rods) and explanatory goals (e.g., understanding why the rodworks the way it does) in a science and engineering curriculum.The role that local interpretation plays in successful implemen-tation became salient by examining the cases in which differentteachers’ strategies achieved similar instructional goals. Indeed,such reinterpretation is inevitable and necessary. Sustainable in-novation requires understanding how and why an innovationworks within a setting over time and across settings (Brown &Campione, 1996), and generating heuristics for those interestedin enacting innovations in their own local contexts.

Relationships BetweenDesign-Based Researchand OtherMethodologies

Design-based research has re-cently been described as a po-tentially fruitful methodologyfor generating causal accountsof learning and instruction thatcould form the basis for system-atic, randomized clinical trials(Levin & O’Donnell, 1999;NRC, 2002). We see design-based research as raising impor-tant questions for research

applied to practice and for research methods, generally. However,randomized trials are not necessarily the appropriate end goal ofour research approach; we do not understand issues of contextwell enough yet to guarantee that randomized trials are the bestmeans to answer the questions we care about. The use of ran-domized trials may hinder innovation studies by prematurelyjudging the efficacy of an intervention. Additionally, random-ized trials may systematically fail to account for phenomena thatviolate this method’s basic assumptions—that is, phenomenathat are contextually dependent or those that result from the in-teraction of dozens, if not hundreds, of factors. Indeed, such phe-nomena are precisely what educational research most needs toaccount for in order to have application to educational practice.We would suggest, however, that design-based research can gen-erate plausible causal accounts because of its focus on linkingprocesses to outcomes in particular settings, and can productivelybe linked with controlled laboratory experiments or randomizedclinical trials (cf. Brown, 1992) by assisting in the identification ofrelevant contextual factors, aiding in identification of mechanisms(not just relationships), and enriching our understanding of thenature of the intervention itself. For example, the CoMPASS

Importantly, design-based

research goes beyond

merely designing and

testing particular

interventions.

at UNIV OF NEW HAMPSHIRE on February 22, 2013http://er.aera.netDownloaded from

project on computer-based conceptual representations in sci-ence learning illustrates one approach to linking design-basedresearch and more traditional studies through “informing cycles”(Puntambekar, 2002). Though initial work showed learning ef-fects of the conceptual representations, alternation between field-based implementation studies and controlled experimental workhelped refine understanding of the important features that affectstudents’ use of these representations.

We do not claim that there is a single design-based researchmethod, but the overarching, explicit concern in design-based re-search for using methods that link processes of enactment to out-comes has power to generate knowledge that directly applies toeducational practice. The value of attending to context is notsimply that it produces a better understanding of an interven-tion, but also that it can lead to improved theoretical accounts ofteaching and learning. In this sense, design-based research differsfrom evaluation research in the ways context and interventionsare problematized.

In traditional evaluation, an “intervention”—an instructionalprogram, a textbook, or a policy—is measured against a set ofstandards (Worthen, Sanders, & Fitzpatrick, 1996). During for-mative evaluation, iterative cycles of development, implementa-tion, and study allow the designer to gather information abouthow an intervention is or is not succeeding in ways that mightlead to better design. Then the intervention is “frozen,” and therigorous summative evaluation begins. Evaluators often concep-tualize context as a set of factors that are independent of the in-tervention itself but that may influence its effects. Like formativeevaluation, design-based research uses mixed methods to analyzean intervention’s outcomes and refine the intervention. Unlikeevaluation research, design-based research views a successful in-novation as a joint product of the designed intervention and thecontext. Hence, design-based research goes beyond perfecting aparticular product. The intention of design-based research in ed-ucation is to inquire more broadly into the nature of learning ina complex system and to refine generative or predictive theoriesof learning. Models of successful innovation can be generatedthrough such work—models, rather than particular artifacts orprograms, are the goal (cf. Brown & Campione, 1996).

Challenges Faced by Design-Based Research Methods

The commitment to using theory-driven design to generate com-plex interventions that can be improved through empirical studyand that can contribute to more basic understanding of the un-derlying theory raises significant challenges. Objectivity, relia-bility, and validity are all necessary to make design-based researcha scientifically sound enterprise, but these qualities are managedin noticeably different ways than in controlled experimentation(e.g., Barab & Kirshner, 2001). Design-based research relies ontechniques used in other research paradigms, like thick descrip-tive datasets, systematic analysis of data with carefully definedmeasures, and consensus building within the field around inter-pretations of data.

By trying to promote objectivity while attempting to facilitatethe intervention, design-based researchers regularly find them-selves in the dual intellectual roles of advocate and critic. Althoughthere are no simple solutions to what we see as a necessary tension

arising from the coupling of empirical research to design, it is pos-sible to employ specific research methods to question the designer-researcher’s tacitly held assumptions. In particular, design-basedresearch typically triangulates multiple sources and kinds of datato connect intended and unintended outcomes to processes ofenactment. In our view, methods that document processes of en-actment provide critical evidence to establish warrants for claimsabout why outcomes occurred.

Complications arise from sustained intervention in messy set-tings. A single, complex intervention (e.g., a 4-week curriculumsequence) might involve hundreds, if not thousands, of discretedesigner, researcher, and teacher decisions—hopefully workingin concert—in an attempt to promote innovative practice. Inthese situations, causality can be difficult to decipher and dis-ambiguate; all possible factors cannot logistically be equally pur-sued; precise replication of an intervention is largely impossible;and emergent phenomena regularly lead to new lines of inquiryinformed by current theories or models of the phenomena. Re-liability of findings and measures can be promoted through tri-angulation from multiple data sources, repetition of analysesacross cycles of enactment, and use (or creation) of standardizedmeasures or instruments.

Validity of findings is often addressed by the partnerships anditeration typical of design-based research, which result in in-creasing alignment of theory, design, practice, and measurementover time. For instance, Hoadley (2002) described how a design-based approach to studying mechanisms for learning via on-linediscussions led to a fundamental rethinking of the theoretical no-tion of social inclusiveness and, with the help of local participantsand teachers, its operationalization in the research program. Thisshift avoided a misinterpretation of data and produced an im-proved learning environment besides.

A logistical challenge for design-based researchers involvesmaintaining a productive collaborative partnership with partici-pants in the research context. Because a single line of researchoften investigates multiple cycles of design, enactment, and study,the work can span years and touch on closely held commitmentsof the researchers and teachers. Indeed, successful examples ofdesign-based research often are conducted within a single settingover a long time (e.g., Linn & Hsi, 2000), and the success of theinnovation and the knowledge gained from its study depend inpart on being able to sustain the partnership between researchersand teachers. There is a trade-off here between the refinement ofa particular innovation to maximize its success, and the general-ization of findings from an ultimately highly refined enactment.The challenge for design-based research is in flexibly developingresearch trajectories that meet our dual goals of refining locallyvaluable innovations and developing more globally usable knowl-edge for the field.

Given the multifaceted nature of the enterprise, a further chal-lenge for design-based research is ensuring that knowledge claimsare used appropriately. We have suggested ways in which design-based research can generate usable knowledge about educationalpractice, but even usable knowledge will not make complex ed-ucational problems simple. Ongoing methodological develop-ment is needed to enhance rigor while respecting the importanceof local context. More importantly, we stress that design-basedresearch should not become a euphemism for “anything goes” re-search or oversimplified interventions.

7JANUARY/FEBRUARY 2003

at UNIV OF NEW HAMPSHIRE on February 22, 2013http://er.aera.netDownloaded from

EDUCATIONAL RESEARCHER8

Prospects for Design-Based Research in Education

We suggest that the value of design-based research should bemeasured by its ability to improve educational practice. We seefour areas where design-based research methods provide themost promise: (a) exploring possibilities for creating novel learn-ing and teaching environments, (b) developing theories of learn-ing and instruction that are contextually based, (c) advancingand consolidating design knowledge, and (d) increasing our ca-pacity for educational innovation. We have seen progress in thearea of developing new insights concerning novel pedagogy inclassroom contexts (e.g., Linn & Hsi, 2000). We should makeconcerted efforts to accumulate the usable knowledge we pro-duce and to cultivate long-term partnerships and infrastructurefor systemic change.

Exploring Possibilities for Novel Learning and Teaching EnvironmentsEfforts to design, use, and do research on educational tools andmaterials in real settings can promote the adoption of innova-tions. They can help researchers and designers understand thereal-world demands placed on designs and on adopters of designs.In addition, pursuing development and enactment through closecollaboration with teachers places them in direct ownership of de-signs. Although design-based research is positioned to addressthese issues, sustaining innovations hinges on our ability to artic-ulate the mechanisms that underlie their success.

Developing Contextualized Theories of Learning and TeachingWe have argued that design-based research methods are of valuein addressing research questions related to the enactment of in-terventions in varying contexts. As a field, educational researchmust develop better theories of the elements of context that mat-ter for the nature of learning and for the implications of policy forlocal educational practices. Design-based research can contributeto such theories through rich accounts of instructional interven-tions and their effects across multiple settings and in multipleareas of instruction.

Constructing Cumulative Design KnowledgeDesign-based research can lead to an understanding of relevantdesign knowledge and practices as they apply to naturalistic set-tings. In design-oriented fields, design knowledge is often char-acterized by common examples, patterns, and principles, and bythe expertise required to apply these generalities in specific set-tings. Currently, design-based research communicates this knowl-edge in many forms, including narratives of planned and enactedinstruction (Hoadley, 2002; Linn & Hsi, 2000), design princi-ples connecting enacted designs to educational outcomes of in-terest (Bell, 2002b), and design patterns abstracted from one ormore settings describing how a designed innovation interactswith settings and evolves (Orrill, 2001). We hope for a scholar-ship of design in education that adopts common communicativeapproaches and links theory to local applied understandings,similar to research in architecture or engineering.

Increasing Human Capacity for InnovationDesign-based research provides numerous opportunities for theexchange of expertise across disciplinary boundaries. Interactionsbetween partners reveal crucial practices that lead to insights

about what occurs when we orchestrate complex interventions inmessy settings (Cobb, 2001). As a natural entailment of designresearch partnerships, the participants frequently learn about thephenomena under study, encounter new theoretical lenses, andacquire experience in conducting and interpreting new analyti-cal techniques (Barab & Kirshner, 2001; Edelson, 2002). Theneed for innovation in education is ongoing, as theories of learn-ing and teaching lead to usable knowledge about and reform ofinstructional practice. Therefore, the development of people whocan apply this knowledge (teachers, administrators, policymakers,and researchers) and who understand and can orchestrate educa-tional change in context should more explicitly drive design-basedresearch partnerships.

Summary

We have argued that design-based research methods can composea coherent methodology that bridges theoretical research and ed-ucational practice. Viewing both the design of an interventionand its specific enactments as objects of research can produce ro-bust explanations of innovative practice and provide principlesthat can be localized for others to apply to new settings. Design-based research, by grounding itself in the needs, constraints, andinteractions of local practice, can provide a lens for understand-ing how theoretical claims about teaching and learning can betransformed into effective learning in educational settings.

NOTE

We thank Larry Bucciarelli, Allan Collins, Valerie Crawford, AndydiSessa, Kris Gutierrez, Eric Hamilton, Yasmin Kafai, Anthony E.Kelly, Carol Lee, Barbara Means, Roy Pea, Brian Reiser, Nora Sabelli,Leona Schauble, Rich Shavelson, Finbarr Sloane, Philip Vahey, andLouise Yarnall, as well as our anonymous reviewers, for valuable feed-back. Klaus Krause, David Perkins, Alexis Whaley, and Jim Kloss pro-vided invaluable editorial help. We especially thank the SpencerFoundation and the funders of our individual research projects for fi-nancial support, although the views expressed here are our own.

AUTHORS

THE DESIGN-BASED RESEARCH COLLECTIVE includes, in alpha-betical order, Eric Baumgartner (University of California at Berkeley andInquirium, LLC); Philip Bell (University of Washington); Sean Brophy(Vanderbilt University); Christopher Hoadley (Pennsylvania State Uni-versity); Sherry Hsi (The Exploratorium); Diana Joseph (University ofChicago); Chandra Orrill (University of Georgia); Sadhana Puntambekar(University of Connecticut); William Sandoval (University of California,Los Angeles); and Iris Tabak (Ben Gurion University of the Negev). Allcontributed to the writing of this article. The Design-Based Research Col-lective is a group of faculty and researchers founded to examine, improve,and practice design-based research methods in education. The group’smembers all blend research on learning and the design of educational in-terventions. The Collective is funded by an Advanced Studies Institutegrant from the Spencer Foundation to Christopher Hoadley at Pennsyl-vania State University. More about the group, its aims, and its membersmay be viewed on-line at http://www.designbasedresearch.org/

Address correspondence to Christopher Hoadley, College of Educa-tion and School of Information Sciences & Technology, PennsylvaniaState University, 314D Keller Building, University Park, PA 16802;dbrc@tophe.net.

Manuscript received March 25, 2002Revisions received November 6, 2002

Accepted November 7, 2002

at UNIV OF NEW HAMPSHIRE on February 22, 2013http://er.aera.netDownloaded from