reproductions supplied by edrs are the best that can be made · blair johnson alisa kenny helen s....

DOCUMENT RESUME

ED 444 126 CS 014 072

TITLE Teaching Children To Read: An Evidence-Based Assessment ofthe Scientific Research Literature on Reading and ItsImplications for Reading Instruction.

INSTITUTION National Reading Panel, Bethesda, MD.SPONS AGENCY National Inst. of Child Health and Human Development (NIH),

Bethesda, MD.REPORT NO NIH-00-4769PUB DATE 2000-04-00NOTE 36p.; See CS 014 073-075 for other "Teaching Children To

Read" document.AVAILABLE FROM Web site: http://www.NationalReadingPanel.org/.PUB TYPE Information Analyses (070) Reports - Descriptive (141)EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Educational Technology; Elementary Education; *Instructional

Effectiveness; Literature Reviews; *Phonics; *ReadingComprehension; *Reading Instruction; *Reading Research;Research Needs; Teacher Education

IDENTIFIERS Phonemic Awareness; *Reading Fluency

ABSTRACTThis report summarizes the work and findings of the National

Reading Panel, charged by Congress to assess the status of research-basedknowledge, including the effectiveness of various approaches to teachingchildren to read. The introduction to the report discusses the charge to thepanel by Congress, establishment of the panel, the panel's approach toachieving its objectives of its charge and initial topic selection, regionalpublic hearings, and adoption of topics studied by the panel. The report thenpresents a methodological overview and the findings and determinations of theNational Reading Panel by topic areas: alphabetics (phonemic awarenessinstruction and phonics instruction); fluency; comprehension (vocabularyinstruction, text comprehension instruction, and teacher preparation andcomprehension strategies instruction); teacher education and readinginstruction; and computer technology and reading instruction. The reportconcludes with a discussion of research needs ("next steps") and reflectionsof the work of the panel. An addendum discusses the processes applied to theselection, review, and analysis of research relevant to reading instruction.(RS)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

N

U.)

REP .11) RT OF THE

NationalReading

PanelTEACHING CHILDREN TO READ

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.Minor changes have boen made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

(N

tt

An Evidence-Based Assessment of the

Scientific Research Literature on Reading and

Its Implications for Reading Instruction

kise copy AV

NATIONAL INSTITUTE OF CHILD HEALTH AND HUMAN DEVELOPMENT

National Institutes of Health

r-%

NationalReading

PanelTEACHING CHILDREN TO READ:

An Evidence-Based Assessmentof the Scientific Research Literature

on Reading and Its Implicationsfor Reading Instruction

3

Acknowledgments

The National Reading Panel wishes to express its gratitude to the following individuals for their contributionsto its effort.

Marilyn Adams Ed Bouchard Harris Cooper Gerald DuffyMichelle Eidlitz Barbara Foorman David Francis Ester HalberstamBlair Johnson Alisa Kenny Helen S. Kim Mad olaine LimbosKhalil Nourani Simone Nunes Elizabeth S. Pang Joan PagnuccoMichael Pressley David Reinking Scott J. Ross Barbara SchusterRobin Sidhu Steven Stahl Maggie Toplak Zoreh Yaghoubzadeh

Report of the National Reading Panel ii

Members of the National Reading Panel

Donald N. Langenberg, Ph.D., Chair

Gloria Correro, Ed.D.Gwenette Ferguson, M.Ed.Michael L. Kamil, Ph.D.Si. Samuels, Ed.D.Sally E. Shaywitz, M.D.Joanna Williams, Ph.D.Joanne Yatvin, Ph.D.

AlphabeticsLinnea Ehri, ChairGloria CorreroTimothy ShanahanDale WillowsJoanne Yatvin

Linnea Ehri, Ph.D.Norma Garza, C.P.A.Cora Bagley Marrett, Ph.D.Timothy Shanahan, Ph.D.Thomas Trabasso, Ph.D.Dale Willows, Ph.D.

MEMBERS OF THE NATIONAL READING PANEL SUBGROUPS

MethodologyTimothy Shanahan, Co-ChairSally E. Shaywitz, Co-Chair

ComprehensionMichael L. Kamil, ChairGwenette FergusonNorma GarzaThomas TrabassoJoanna Williams

Teacher EducationGloria Corerro, Co-ChairMichael L. Kamil, Co-ChairGwenette FergusonNorma GarzaCora Bagley Marrett

FluencyS.J. Samuels, Co-ChairTimothy Shanahan, Co-ChairSally E. Shaywitz

Technology/Next StepsMichael L. Kamil, ChairDonald N. Langenberg

STAFF OF THE NATIONAL READING PANEL

F. William Dommel, Jr., J.D., Executive Director

Vinita Chhabra, M.Ed., Research ScientistJudith Rothenberg, SecretaryJaimee Nusbacher, Meeting Manager

Mary E. McCarthy, Ph.D., Senior Staff PsychologistStephanne Player, Support StaffPatrick Riccards, Senior Advisor

5iii Report of the National Reading Panel

Table of Contents

Acknowledgments ii

Members of the National Reading Panel iii

Introduction 1

Congressional Charge 1

Establishment of the National Reading Panel 1

NRP Approach to Achieving the Objectives of Its Chargeand Initial Topic Selection 1

Regional Public Hearings 2Adoption of Topics To Be Studied 2

Methodological Overview 5

Findings and Determinations of the National Reading Panel by Topic Areas 7Alphabetics 7

Phonemic Awareness Instruction 7Phonics Instruction 8

Fluency 11Comprehension 13

Vocabulary Instruction 13Text Comprehension Instruction 14Teacher Preparation and Comprehension Strategies Instruction 15

Teacher Education and Reading Instruction .16Computer Technology and Reading Instruction 17

Next Steps 19

Reflections 21

Addendum 25Methodology: Processes Applied to the Selection, Review, and

Analysis of Research Relevant to Reading Instruction 27

(Report of the National Reading Panel: Reports of the Subgroups provides complete and extensive descriptionsof information presented in this Report.)

v Report of the National Reading Panel

Introduction

Congressional ChargeIn 1997, Congress asked the "Director of theNational Institute of Child Health and HumanDevelopment (NICHD), in consultation with theSecretary of Education, to convene a national panel toassess the status of research-based knowledge,including the effectiveness of various approaches toteaching children to read." This panel was chargedwith providing a report that "should present thepanel's conclusions, an indication of the readiness forapplication in the classroom of the results of thisresearch, and, if appropriate, a strategy for rapidlydisseminating this information to facilitate effectivereading instruction in the schools. If found warranted,the panel should also recommend a plan for additionalresearch regarding early reading development andinstruction."

Establishment of theNational Reading PanelIn response to this Congressional request, theDirector of NICHD, in consultation with the Secretaryof Education, constituted and charged a NationalReading Panel (the NRP or the Panel). The NRPcomprised 14 individuals, including (as specified byCongress) "leading scientists in reading research,representatives of colleges of education, readingteachers, educational administrators, and parents."The original charge to the NRP asked that a finalreport be submitted by November 1998. When thePanel began its work, it quickly became apparent thatthe Panel could not respond properly to its chargewithin that time constraint. Permission was sought andreceived to postpone the report's submissiondeadline. A progress report was submitted toCongress in February 1999. The informationprovided in the NRP Progress Report, this Report ofthe National Reading Panel, and the Report of the

National Reading Panel: Reports of the Subgroupsreflect the findings and determinations of the NationalReading Panel.

NRP Approach to Achieving theObjectives of Its Charge and InitialTopic SelectionThe charge to the NRP took into account thefoundational work of the National Research Council(NRC) Committee on Preventing Reading Difficultiesin Young Children (Snow, Burns, & Griffin, 1998).The NRC report is a consensus document based onthe best judgments of a diverse group of experts inreading research and reading instruction. The NRCCommittee identified and summarized researchliterature relevant to the critical skills, environments,and early developmental interactions that areinstrumental in the acquisition of beginning readingskills. The NRC Committee did not specificallyaddress "how" critical reading skills are mosteffectively taught and what instructional methods,materials, and approaches are most beneficial forstudents of varying abilities.

In order to build upon and expand the work of theNRC Committee, the NRP first developed anobjective research review methodology. The Panelthen applied this methodology to undertakecomprehensive, formal, evidence-based analyses ofthe experimental and quasi-experimental researchliterature relevant to a set of selected topics judged tobe of central importance in teaching children to read.An examination of a variety of public databases byPanel staff revealed that approximately 100,000research studies on reading have been published since1966, with perhaps another 15,000 appearing beforethat time. Obviously, it was not possible for a panel ofvolunteers to examine critically this entire body ofresearch literature. Selection of prioritized topics wasnecessitated by the large amount of published reading

1 Report of the National Reading Panel7

Teaching Children to Read

research literature relevant to the Panel's charge todetermine the effectiveness of reading instructionalmethods and approaches. A screening process wastherefore essential.

The Panel's initial screening task involved selection ofthe set of topics to be addressed. Recognizing thatthis selection would require the use of informedjudgment, the Panel chose to begin its work bybroadening its understanding of reading issues througha thorough analysis of the findings of the NRC report,Preventing Reading Difficulties in Young Children(Snow, Bums, & Griffin, 1998). Early in itsdeliberations the Panel made a tentative decision toestablish subgroups of its members and to assign toeach of them one of the major topic areas designatedby the NRC Committee as central to learning toreadAlphabetics, Fluency, and Comprehension.

Regional Public HearingsAs part of its information gathering, the Panel publiclyannounced, planned, and held regional hearings inChicago, IL (May 29, 1998), Portland, OR (June 5,1998), Houston, TX (June 8, 1998), New York, NY(June 23, 1998), and Jackson, MS (July 9, 1998).The Panel believed that it would not have beenpossible to accomplish the mandate of Congresswithout first hearing directly from consumers of thisinformationteachers, parents, students, andpolicymakersabout their needs and theirunderstanding of the research. Although the regionalhearings were not intended as a substitute for scientificresearch, the hearings gave the Panel an opportunityto listen to the voices of those who will need toconsider implementation of the Panel's findings anddeterminations. The regional hearings gave membersa clearer understanding of the issues important to thepublic.

As a result of these hearings, the Panel received oraland written testimony from approximately 125individuals or organizations representing citizensteachers, parents, students, university faculty,educational policy experts, and scientistswho wouldbe the ultimate users and beneficiaries of the research-derived findings and determinations of the Panel.

At the regional hearings, several key themes wereexpressed repeatedly:

The importance of the role of parents and otherconcerned individuals, especially in providingchildren with early language and literacyexperiences that foster reading development;

The importance of early identification andintervention for all children at risk for readingfailure;

The importance of phonemic awareness, phonics,and good literature in reading instruction and theneed to develop a clear understanding of how bestto integrate different reading approaches toenhance the effectiveness of instruction for allstudents;

The need for clear, objective, and scientificallybased information on the effectiveness of differenttypes of reading instruction and the need to havesuch research inform policy and practice;

The importance of applying the highest standardsof scientific evidence to the research reviewprocess so that conclusions and determinations arebased on findings obtained from experimentalstudies characterized by methodological rigor withdemonstrated reliability, validity, replicability, andapplicability;

The importance of the role of teachers, theirprofessional development, and their interactionsand collaborations with researchers, which shouldbe recognized and encouraged; and

The importance of widely disseminating theinformation that is developed by the Panel.

Adoption of Topics To Be StudiedFollowing the regional hearings, the Panel considered,discussed, and debated several dozen possible topicareas and then settled on the following topics forintensive study:

Alphabetics

Phonemic Awareness Instruction

Phonics Instruction

Report of the National Reading Panel 2

"-I,

An Evidence-Based Assessment

Fluency

Comprehension

Vocabulary Instruction

Text Comprehension Instruction

Teacher Preparation and ComprehensionStrategies Instruction

Teacher Education and Reading Instruction

Computer Technology and Reading Instruction.

In addition, because of the concern voiced by thepublic at the regional hearings that the higheststandards of scientific evidence be applied in theresearch review process, the methodology subgroupwas tasked to develop a research review processincluding specific review criteria.

Each topic and subtopic became the subject of thework of a subgroup composed of one or more Panelmembers. Some Panel members served on more thanone subgroup. The subgroups formulated sevenbroad questions to guide their efforts in meeting theCongressional charge of identifying effectiveinstructional reading approaches and determining theirreadiness for application in the classroom:

1. Does instruction in phonemic awareness improvereading? If so, how is this instruction bestprovided?

2. Does phonics instruction improve readingachievement? If so, how is this instruction bestprovided?

3. Does guided oral reading instruction improvefluency and reading comprehension? If so, how isthis instruction best provided?

4. Does vocabulary instruction improve readingachievement? If so, how is this instruction bestprovided?

5. Does comprehension strategy instruction improvereading? If so, how is this instruction bestprovided?

6. Do programs that increase the amount ofchildren's independent reading improve readingachievement and motivation? If so, how is thisinstruction best provided?

7. Does teacher education influence how effectiveteachers are at teaching children to read? If so,how is this instruction best provided?

Each subgroup also generated several subordinatequestions to address within each of the majorquestions. It should be made clear that the Panel didnot consider these questions and the instructionalissues that they represent to be the only topics ofimportance in learning to read. The Panel's silence onother topics should not be interpreted as indicating thatother topics have no importance or that improvementin those areas would not lead to greater readingachievement. It was simply the sheer number ofstudies identified by Panel staff relevant to reading(more than 100,000 published since 1966 and morethan 15,000 prior to 1966) that precluded anexhaustive analysis of the research in all areas ofpotential interest.

The Panel also did not address issues relevant tosecond language learning, as this topic was beingaddressed in detail in a new, comprehensive NICHD/OERI (Office of Educational Research andImprovement) research initiative. The questionspresented above bear on instructional topics ofwidespread interest in the field of reading educationthat have been articulated in a wide range of theories,research studies, instructional programs, curricula,assessments, and educational policies. The Panelelected to examine these and subordinate questionsbecause they currently reflect the central issues inreading instruction and reading achievement. Themethodological processes described in the nextsection guided the Panel's examination and analysis ofthe extant research.

3 9Report of the National Reading Panel

Methodological Overview

In what may be its most important action, the Panelthen developed and adopted a set of rigorous researchmethodological standards. (See the methodologyadopted by the Panel and printed as an addendum tothis Report.) These standards guided the screening ofthe research literature relevant to each topic areaaddressed by the Panel. This screening processidentified a final set of experimental or quasi-experimental research studies that were then subjectedto detailed analysis. The evidence-basedmethodological standards adopted by the Panel areessentially those normally used in research studies ofthe efficacy of interventions in psychological andmedical research. These include behaviorally basedinterventions, medications, or medical proceduresproposed for use in the fostering of robust health andpsychological development and the prevention ortreatment of disease.

It is the view of the Panel that the efficacy of materialsand methodologies used in the teaching of reading andin the prevention or treatment of reading disabilitiesshould be tested no less rigorously. However, suchstandards have not been universally accepted or usedin reading education research. Unfortunately, only asmall fraction of the total reading research literaturemet the Panel's standards for use in the topic analyses.

The research literature screening process proceededessentially as follows. For each topic, an initial pool ofcandidate studies was created by searching a minimumof two databases (PsycINFO and ERIC) for studyreports relevant to the topic. To be included in thedatabase, studies had to measure reading as anoutcome. Reading was defined to include severalbehaviors such as the following: reading real words inisolation or in context, reading pseudowords that canbe pronounced but have no meaning, reading textaloud or silently, and comprehending text that is read

silently or orally. From the pool produced by theelectronic searches of the databases, those studieswere selected that met the following criteria:

Published in English in a refereed journal;

Focused on children's reading development in theage/grade range from preschool to grade 12; and

Used an experimental or quasi-experimentaldesign with a control group or a multiple-baselinemethod.

Those studies meeting the above criteria formed theset of studies subjected to further analysis. The nextstep was to code each study for several characteristicsincluding the following:

Characteristics of study participants (age;demographics; cognitive, academic, andbehavioral characteristics);

Study interventions, described in sufficient detail toallow for replicability, including how long theinterventions lasted and how long the effectslasted;

Study methods, with sufficient description to allowjudgments about how instruction fidelity wasinsured; and

Nature of the outcome measures and whether theywere described fully.

For each study meeting the above criteria, relevantreported statistics were coded in a standardizedformat and analyzed. For several topics, the numberof studies meeting criteria was sufficient to permit aformal statistical meta-analySis, including calculation ofeffect sizes. For others, a full meta-analysis could notbe carried out. Where there were too few studies thatsatisfied the Panel's criteria to permit a meta-analysis,the Panel made a decision to conduct a moresubjective qualitative analysis to provide the bestpossible information about an instructional readingapproach or program.

51 0

Report of the National Reading Panel


With this information as background, this Report isorganized into sections to provide an overview of themajor findings and determinations achieved by theNRP in the areas of alphabetics (phonemic awarenessinstruction and phonics instruction), fluency,comprehension (vocabulary instruction, text

comprehension instruction, and teacher preparationand comprehension strategies instruction), teachereducation and reading instruction, computertechnology and reading instruction, and next steps.This Report concludes with some reflections on theNRP process and products.


Findings and Determinations of theNational Reading Panel by Topic Areas

Alphabetics

Phonemic Awareness Instruction

Phonemes are the smallest units composing spokenlanguage. For example, the words "go" and "she"each consist of two sounds or phonemes. Phonemesare different from letters that represent phonemes inthe spellings of words. Instruction in phonemicawareness (PA) involves teaching children to focus onand manipulate phonemes in spoken syllables andwords. PA instruction is frequently confused withphonics instruction, which entails teaching studentshow to use letter-sound relations to read or spellwords. PA instruction qualifies as phonics instructionwhen it involves teaching children to blend or segmentthe sounds in words using letters. However, childrenmay be taught to manipulate sounds in speech withoutany letters as well; this does not qualify as phonicsinstruction. PA is also frequently confused withauditory discrimination, which refers to the ability torecognize whether two spoken words are the same ordifferent. These distinctions are explained in detail inthe section devoted to phonemic awareness instructionin the Report of the National Reading Panel: Reportsof the Subgroups.

There are several reasons why the NRP selected PAinstruction for review and analysis. First, correlationalstudies have identified PA and letter knowledge as thetwo best school-entry predictors of how well childrenwill learn to read during the first 2 years of instruction.Such evidence suggests the potential importance of PAtraining in the development of reading skills. Second,many experimental studies have been carried out toevaluate the effectiveness of PA training in facilitatingreading acquisition. Third, there is currently muchinterest in PA training programs among teachers,principals, parents, and publishers because of claimsabout their value in improving children's ability to learnto read.

The initial literature search for studies relevant to PAinstruction and training identified 1,962 citations.Following initial review, the Panel identified and furtherreviewed 78 studies that met the general NRPresearch methodology criteria. However, on detailedexamination, only 52 studies satisfied the more specificNRP research methodology criteria. From these 52studies, 96 comparisons of treatment and controlgroups were derived. Data from these comparisonswere then entered into a meta-analysis todeterminetreatment effect sizes.

Findings and DeterminationsThe results of the meta-analysis were impressive.Overall, the findings showed that teaching children tomanipulate phonemes in words was highly effectiveunder a variety of teaching conditions with a variety oflearners across a range of grade and age levels andthat teaching phonemic awareness to childrensignificantly improves their reading more thaninstruction that lacks any attention to PA.

Specifically, the results of the experimental studies ledthe Panel to conclude that PA training was the causeof improvement in students' phonemic awareness,reading, and spelling following training. The findingswere replicated repeatedly across multipleexperiments and thus provide converging evidence forcausal claims. While PA training exerted strong andsignificant effects on reading and spelling development,it did not have an impact on children's performance onmath tests. This indicates that halo/Hawthorne(novelty) effects did not explain the findings and thatindeed the training effects were directly connectedwith and limited to the targeted domain under study.Importantly, the effects of PA instruction on readinglasted well beyond the end of training. Children ofvarying abilities improved their PA and their readingskills as a function of PA training.

7 Report of the National Reading Panel


PA instruction also helped normally achieving childrenlearn to spell, and the effects lasted well beyond theend of training. However, the instruction was noteffective for improving spelling in disabled readers.This is consistent with other research showing thatdisabled readers have difficulty learning how to spell.

Programs in all of the studies provided explicitinstruction in phonemic awareness. Specifically, thecharacteristics of PA training found to be mosteffective in enhancing PA, reading, and spelling skillsincluded explicitly and systematically teaching childrento manipulate phonemes with letters, focusing theinstruction on one or two types of phonememanipulations rather than multiple types, and teachingchildren in small groups.

PA instruction is ready for implementation in theclassroom, but teachers should keep in mind severalcautions. First, PA training does not constitute acomplete reading program. Rather, it provideschildren with essential foundational knowledge in thealphabetic system. It is one necessary instructionalcomponent within a complete and integrated readingprogram. Several additional competencies must beacquired as well to ensure that children will learn toread and write. Second, there are many ways toteach PA effectively. In implementing PA instruction,teachers need to evaluate the methods they use againstmeasured success in their own students. Third, themotivation of both students and their teachers is acritical ingredient of success. Research has notspecifically focused on this.

Phonics Instruction

Phonics instruction is a way of teaching reading thatstresses the acquisition of letter-soundcorrespondences and their use in reading and spelling.The primary focus of phonics instruction is to helpbeginning readers understand how letters are linked tosounds (phonemes) to form letter-soundcorrespondences and spelling patterns and to helpthem learn how to apply this knowledge in theirreading. Phonics instruction may be providedsystematically or incidentally. The hallmark of a

Phonics Instructional Approaches

Analogy PhonicsTeaching studentsunfamiliar words by analogy to knownwords (e.g., recognizing that the rimesegment of an unfamiliar word is identical tothat of a familiar word, and then blending theknown rime with the new word onset, suchas reading brick by recognizing that -ick iscontained in the known word kick, orreading stump by analogy to jump).

Analytic PhonicsTeaching students toanalyze letter-sound relations in previouslylearned words to avoid pronouncing soundsin isolation.

Embedded PhonicsTeaching studentsphonics skills by embedding phonicsinstruction in text reading, a more implicitapproach that relies to some extent onincidental learning.

Phonics through SpellingTeachingstudents to segment words into phonemesand to select letters for those phonemes(i.e., teaching students to spell wordsphonemically).

Synthetic PhonicsTeaching studentsexplicitly to convert letters into sounds(phonemes) and then blend the sounds toform recognizable words.

systematic phonics approach or program is that asequential set of phonics elements is delineated andthese elements are taught along a dimension ofexplicitness depending on the type of phonics methodemployed. Conversely, with incidental phonicsinstruction, the teacher does not follow a plannedsequence of phonics elements to guide instruction buthighlights particular elements opportunistically whenthey appear in text.

Report of the National Reading Panel 8 1 3


Types of Phonics Instructional Methods andApproachesThe sidebar depicts several different types of phonicsinstructional approaches that vary according to the unitof analysis or how letter-sound combinations arerepresented to the student. For example, in syntheticphonics approaches, students are taught to link anindividual letter or letter combination with itsappropriate sound and then blend the sounds to formwords. In analytic phonics, students are first taughtwhole word units followed by systematic instructionlinking the specific letters in the word with theirrespective sounds. Phonics instruction can also varywith respect to the explicitness by which the phonicelements are taught and practiced in the reading oftext. For example, many synthetic phonicsapproaches use direct instruction in teaching phonicscomponents and provide opportunities for applyingthese skills in decodable text formats characterized bya controlled vocabulary. On the other hand,embedded phonics approaches are typically lessexplicit and use decodable text for practice lessfrequently, although the phonics concepts to belearned can still be presented systematically. Thesedistinctions are addressed in detail in the Phonicssubgroup report.

Questions Guiding the NRP Analysis ofPhonics instructionTheNRP examined the research literature concerningphonics instruction to answer the following questions:Does phonics instruction enhance children's success inlearning to read? Is phonics instruction more effectiveat some grade levels than others? Is it beneficial forchildren who are having difficulties learning to read?Does phonics instruction improve all aspects ofreading or just decoding and word-level reading skills?Are some types of phonics instruction more effectivethan others and for which children? Does phonicsinstruction have an impact on children's spelling?

To address these questions the NRP performed aliterature search to identify studies published since1970 that compared phonics instruction to other formsof instruction for their impact on reading ability. The

initial electronic and manual searches identified 1,373studies that appeared relevant to phonics instruction.Evaluation of these studies to determine adherence tothe general and specific NRP research methodologycriteria identified 38 studies from which 66 treatment-control group comparisons were derived. Data fromthese studies were used in a meta-analysis, includingthe calculation of effect sizes.

The meta-analysis indicated that systematic phonicsinstruction enhances children's success in learning toread and that systematic phonics instruction issignificantly more effective than instruction that teacheslittle or no phonics.

Findings and DeterminationsThe meta-analysis revealed that systematic phonicsinstruction produces significant benefits for students inkindergarten through 6th grade and for children havingdifficulty learning to read. The ability to read and spellwords was enhanced in kindergartners who-receivedsystematic beginning phonics instruction. First graderswho were taught phonics systematically were betterable to decode and spell, and they showed significantimprovement in their ability to comprehend text.Older children receiving phonics instruction werebetter able to decode and spell words and to read textorally, but their comprehension of text was notsignificantly improved.

Systematic synthetic phonics instruction (see sidebarfor definition) had a positive and significant effect ondisabled readers' reading skills. These childrenimproved substantially in their ability to read wordsand showed significant, albeit small, gains in theirability to process text as a result of systematicsynthetic phonics instruction. This type of phonicsinstruction benefits both students with learningdisabilities and low-achieving students who are notdisabled. Moreover, systematic synthetic phonicsinstruction was significantly more effective in improvinglow socioeconomic status (SES) children's alphabeticknowledge and word reading skills than instructionalapproaches that were less focused on these initialreading skills.

9 Report of the. National Reading Panel


Across all grade levels, systematic phonics instructionimproved the ability of good readers to spell. Theimpact was strongest for kindergartners anddecreased in later grades. For poor readers, theimpact of phonics instruction on spelling was small,perhaps reflecting the consistent finding that disabledreaders have trouble learning to spell.

Although conventional wisdom has suggested thatkindergarten students might not be ready for phonicsinstruction, this assumption was not supported by thedata. The effects of systematic early phonicsinstruction were significant and substantial inkindergarten and the 1st grade, indicating thatsystematic phonics programs should be implementedat those age and grade levels.

The NRP analysis indicated that systematic phonicsinstruction is ready for implementation in theclassroom. Findings of the Panel regarding theeffectiveness of explicit, systematic phonics instructionwere derived from studies conducted in manyclassrooms with typical classroom teachers and typicalAmerican or English-speaking students from a varietyof backgrounds and socioeconomic levels. Thus, theresults of the analysis are indicative of what can beaccomplished when explicit, systematic phonicsprograms are implemented in today's classrooms.Systematic phonics instruction has been used widelyover a long period of time with positive results, and avariety of systematic phonics programs have proveneffective with children of different ages, abilities, andsocioeconomic backgrounds.

These facts and findings provide converging evidencethat explicit, systematic phonics instruction is avaluable and essential part of a successful classroomreading program. However, there is a need to becautious in giving a blanket endorsement of all kinds ofphonics instruction.

It is important to recognize that the goals of phonicsinstruction are to provide children with key knowledgeand skills and to ensure that they know how to applythat knowledge in their reading and writing. In otherwords, phonics teaching is a means to an end. To be

able to make use of letter-sound information, childrenneed phonemic awareness. That is, they need to beable to blend sounds together to decode words, andthey need to break spoken words into their constituentsounds to write words. Programs that focus too muchon the teaching of letter-sound relations and notenough on putting them to use are unlikely to be veryeffective. In implementing systematic phonicsinstruction, educators must keep the end in mind andensure that children understand the purpose of learningletter sounds and that they are able to apply theseskills accurately and fluently in their daily reading andwriting activities.

Of additional concern is the often-heard call for"intensive, systematic" phonics instruction. Usually theterm "intensive" is not defined. How much is requiredto be considered intensive? In addition, it is not clearhow many months or years a phonics program shouldcontinue. If phonics has been systematically taught inkindergarten and 1st grade, should it continue to beemphasized in 2nd grade and beyond? How longshould single instruction sessions last? How muchground should be covered in a program? Specifically,how many letter-sound relations should be taught, andhow many different ways of using these relations toread and write words should be practiced for thebenefits of phonics to be maximized? These questionsremain for future research.

Another important area is the role of the teacher.Some phonics programs showing large effect sizesrequire teachers to follow a set of specific instructionsprovided by the publisher; while this may standardizethe instructional sequence, it also may reduce teacherinterest and motivation. Thus, one concern is how tomaintain consistency of instruction while stillencouraging the unique contributions of teachers.Other programs require a sophisticated knowledge ofspelling, structural linguistics, or word etymology. Inview of the evidence showing the effectiveness ofsystematic phonics instruction, it is important to ensurethat the issue of how best to prepare teachers to carryout this teaching effectively and creatively is given highpriority.

Report of the National Reading Panel 10 15


Types of Phonics instructional Methods andApproachesThe sidebar depicts several different types of phonicsinstructional approaches that vary according to the unitof analysis or how letter-sound combinations arerepresented to the student. For example, in syntheticphonics approaches, students are taught to link anindividual letter or letter combination with itsappropriate sound and then blend the sounds to formwords. In analytic phonics, students are first taughtwhole word units followed by systematic instructionlinking the specific letters in the word with theirrespective sounds. Phonics instruction can also varywith respect to the explicitness by which the phonicelements are taught and practiced in the reading oftext. For example, many synthetic phonicsapproaches use direct instruction in teaching phonicscomponents and provide opportunities for applyingthese skills in decodable text formats characterized bya controlled vocabulary. On the other hand,embedded phonics approaches are typically lessexplicit and use decodable text for practice lessfrequently, although the phonics concepts to belearned can still be presented systematically. Thesedistinctions are addressed in detail in the Phonicssubgroup report.

Questions Guiding the NRP Analysis ofPhonics InstructionThe NRP examined the research literature concerningphonics instruction to answer the following questions:Does phonics instruction enhance children's success inlearning to read? Is phonics instruction more effectiveat some grade levels than others? Is it beneficial forchildren who are having difficulties learning to read?Does phonics instruction improve all aspects ofreading or just decoding and word-level reading skills?Are some types of phonics instruction more effectivethan others and for which children? Does phonicsinstruction have an impact on children's spelling?

To address these questions the NRP performed aliterature search to identify studies published since1970 that compared phonics instruction to other formsof instruction for their impact on reading ability. The

initial electronic and manual searches identified 1,373studies that appeared relevant to phonics instruction.Evaluation of these studies to determine adherence tothe general and specific NRP research methodologycriteria identified 38 studies from which 66 treatment-control group comparisons were derived. Data fromthese studies were used in a meta-analysis, includingthe calculation of effect sizes.

The meta-analysis indicated that systematic phonicsinstruction enhances children's success in learning toread and that systematic phonics instruction issignificantly more effective than instruction that teacheslittle or no phonics.

Findings and DeterminationsThe meta-analysis revealed that systematic phonicsinstruction produces significant benefits for students inkindergarten through 6th grade and for children havingdifficulty learning to read. The ability to read and spellwords was enhanced in kindergartners who-receivedsystematic beginning phonics instruction. First graderswho were taught phonics systematically were betterable to decode and spell, and they showed significantimprovement in their ability to comprehend text.Older children receiving phonics instruction werebetter able to decode and spell words and to read textorally, but their comprehension of text was notsignificantly improved.

Systematic synthetic phonics instruction (see sidebarfor definition) had a positive and significant effect ondisabled readers' reading skills. These childrenimproved substantially in their ability to read wordsand showed significant, albeit small, gains in theirability to process text as a result of systematicsynthetic phonics instruction. This type of phonicsinstruction benefits both students with learningdisabilities and low-achieving students who are notdisabled. Moreover, systematic synthetic phonicsinstruction was significantly more effective in improvinglow socioeconomic status (SES) children's alphabeticknowledge and word reading skills than instructionalapproaches that were less focused on these initialreading skills.



Reading practice is generally recognized as animportant contributor to fluency. Two instructionalapproaches, each of which has several variations, havetypically been used to teach reading fluency. One,guided repeated oral reading, encourages students toread passages orally with systematic and explicitguidance and feedback from the teacher. The other,independent silent reading, encourages students toread silently on their own, inside and outside theclassroom, with minimal guidance or feedback.

Guided Oral Reading

The NRP conducted an initial series of electronicliterature searches and identified 364 studiespotentially relevant to the effects of guided oral readinginstructional practices. Of these, 16 studies met theNRP research methodology criteria and were includedin a meta-analysis, and 21 additional studies met thecriteria but could not be included in the meta-analysisalthough they were used in the qualitativeinterpretation of the efficacy of these instructionalmethods.

Findings and DeterminationsOn the basis of a detailed analysis of the availableresearch that met NRP methodological criteria, thePanel concluded that guided repeated oral readingprocedures that included guidance from teachers,peers, or parents had a significant and positive impacton word recognition, fluency, and comprehensionacross a range of grade levels. These studies wereconducted in a variety of classrooms in both regularand special education settings with teachers usingwidely available instructional materials. This suggeststhe classroom readiness of guided oral reading andrepeated reading procedures. These results also applyto all studentsgood readers as well as thoseexperiencing reading difficulties. Nevertheless, therewere important gaps in the research. In particular, thePanel could find no multiyear studies providinginformation on the relationship between guided oralreading and the emergence of fluency.

Independent Silent Reading

There has been widespread agreement in the literaturethat encouraging students to engage in wide,independent, silent reading increases readingachievement. Literally hundreds of correlationalstudies find that the best readers read the most andthat poor readers read the least. These correlationalstudies suggest that the more that children read, thebetter their fluency, vocabulary, and comprehension.However, these findings are correlational in nature,and correlation does not imply causation. No doubt, itcould be that the more that children read, the moretheir reading skills improve, but it is also possible thatbetter readers simply choose to read more.

In order to address this issue of causation, the panelexamined the specific impact that encouraging studentsto read more has on fluency, vocabulary development,and reading comprehension. The studies that wereidentified that address this issue were characterized bythree major features. First, the studies emphasizedsilent reading procedures with students reading ontheir own with little or no specific feedback. Second,the studies did not directly assess fluency or the actualincrease in the amount of reading due to theinstructional procedures. Rather, only changes invocabulary and/or comprehension were typicallymeasured as outcomes rather than increases in fluencythat could be expected from the increased readingpractice. Third, very few studies that examined theeffect of independent silent reading on readingachievement could meet the NRP research reviewmethodology criteria (n =14), and these studies variedwidely in their methodological quality and the readingoutcome variables measured. Thus, a meta-analysiscould not be conducted. Rather, the 14 studies wereexamined individually and in detail to identifyconverging trends and findings in the data.

Findings and DeterminationsWith regard to the efficacy of having students engagein independent silent reading with minimal guidance orfeedback, the Panel was unable to find a positiverelationship between programs and instruction that



encourage large amounts of independent reading andimprovements in reading achievement, includingfluency. In other words, even though encouragingstudents to read more is intuitively appealing, there isstill not sufficient research evidence obtained fromstudies of high methodological quality to support theidea that such efforts reliably increase how muchstudents read or that such programs result in improvedreading skills. Given the extensive use of thesetechniques, it is important that such research beconducted.

It should be made clear that these findings do notnegate the positive influence that independent silentreading may have on reading fluency, nor do thefindings negate the possibility that wide independentreading significantly influences vocabularydevelopment and reading comprehension. Rather,there are simply not sufficient data from well-designedstudies capable of testing questions of causation tosubstantiate causal claims. The available data dosuggest that independent silent reading is not aneffective practice when used as the only type ofreading instruction to develop fluency and otherreading skills, particularly with students who have notyet developed critical alphabetic and word readingskills. In sum, methodologically rigorous researchdesigned to assess the specific influences thatindependent silent reading practices have on readingfluency and other reading skills and the motivation toread has not yet been conducted.

ComprehensionComprehension is critically important to thedevelopment of children's reading skills and thereforeto the ability to obtain an education. Indeed, readingcomprehension has come to be the "essence ofreading" (Durkin, 1993), essential not only toacademic learning in all subject areas but to lifelonglearning as well. In carrying out its analysis of theextant research in reading comprehension, the NRPnoted three predominant themes in the research on thedevelopment of reading comprehension skills. First,reading comprehension is a complex cognitive processthat cannot be understood without a clear description

of the role that vocabulary development andvocabulary instruction play in the understanding ofwhat has been read. Second, comprehension is anactive process that requires an intentional andthoughtful interaction between the reader and the text.Third, the preparation of teachers to better equipstudents to develop and apply reading comprehensionstrategies to enhance understanding is intimately linkedto students' achievement in this area. Because thesethree themes serve as the foundation for understandinghow best to help teachers develop students'comprehension abilities, the extant research relevant tovocabulary instruction, to text comprehensioninstruction, and to the preparation of teachers to teachreading comprehension strategies was examined indetail by the NRP. The major findings anddeterminations of the Panel for each of these threesubareas are provided next.

Vocabulary Instruction

The importance of vocabulary knowledge has longbeen recognized in the development of reading skills.As early as 1924, researchers noted that growth inreading power means continuous growth in wordknowledge (Whipple, 1925). Vocabulary is criticallyimportant in oral reading instruction. There are twotypes of vocabularyoral and print. A reader whoencounters a strange word in print can decode theword to speech. If it is in the reader's oralvocabulary, the reader will be able to understand it. Ifthe word is not in the reader's oral vocabulary, thereader will have to determine the meaning by othermeans, if possible. Consequently, the larger thereader's vocabulary (either oral or print), the easier itis to make sense of the text.

To determine how vocabulary can best be taught andrelated to the reading comprehension process, theNRP examined more than 20,000 research citationsidentified through electronic and manual literaturesearches. From this set, citations were removed ifthey did not meet prespecified criteria: if they werenot reports of research, if they were not reportingexperimental or quasi-experimental studies, if they

13



were not published in English, or if they dealtexclusively with learning disabled or other specialpopulations, including second-language learners.Comprehensive review of the remaining set of studiesaccording to the NRP review criteria identified 50studies for further evaluation. Further analysis andcoding of these studies indicated that a formal meta-analysis could not be conducted because there was asmall number of research studies in vocabularyinstruction dealing with a relatively large number ofvariables. There are recent published meta-analysesfor some selected variables, and it was decided not toduplicate those efforts. Also, a substantial amount ofpublished research on vocabulary instruction did notmeet NRP research methodology criteria. Becausethe Panel wanted to glean as much information aspossible from the studies identified in the searches, thevocabulary instruction database was reviewed fortrends across studies, even though formal meta-analyses could not be conducted. Fifty studies datingfrom 1979 to the present were reviewed in detail.There were 21 different methods represented in thesestudies.

Findings and DeterminationsThe studies reviewed suggest that vocabularyinstruction does lead to gains in comprehension, butthat methods must be appropriate to the age andability of the reader. The use of computers invocabulary instruction was found to be more effectivethan some traditional methods in a few studies. It isclearly emerging as a potentially valuable aid toclassroom teachers in the area of vocabularyinstruction. Vocabulary also can be learnedincidentally in the context of storybook reading or inlistening to others. Learning words before reading atext also is helpful. Techniques such as taskrestructuring and repeated exposure (including havingthe student encounter words in various contexts)appear to enhance vocabulary development. Inaddition, substituting easy words for more difficultwords can assist low-achieving students.

The findings on vocabulary yielded several specificimplications for teaching reading. First, vocabularyshould be taught both directly and indirectly.Repetition and multiple exposures to vocabulary itemsare important. Learning in rich contexts, incidentallearning, and use of computer technology all enhancethe acquisition of vocabulary. Direct instruction shouldinclude task restructuring as necessary and shouldactively engage the student. Finally, dependence on asingle vocabulary instruction method will not result inoptimal learning.

While much is known about the importance ofvocabulary to success in reading, there is little researchon the best methods or combinations of methods ofvocabulary instruction and the measurement ofvocabulary growth and its relation to instructionmethods.

Text Comprehension Instruction

Comprehension is defined as "intentional thinkingduring which meaning is constructed throughinteractions between text and reader" (Harris &Hodges, 1995). Thus, readers derive meaning fromtext when they engage in intentional, problem solvingthinking processes. The data suggest that textcomprehension is enhanced when readers activelyrelate the ideas represented in print to their ownknowledge and experiences and construct mentalrepresentations in memory.

The rationale for the explicit teaching ofcomprehension skills is that comprehension can beimproved by teaching students to use specific cognitivestrategies or to reason strategically when theyencounter barriers to understanding what they arereading. Readers acquire these strategies informally tosome extent, but explicit or formal instruction in theapplication of comprehension strategies has beenshown to be highly effective in enhancing

understanding. The teacher generally demonstratessuch strategies for students until the students are ableto carry them out independently.



The literature search identified 453 studies thataddressed issues and topics relevant to textcomprehension since 1980. Studies publishedbetween 1970 and 1979 were added if they were ofparticular relevance, resulting in 481 studies that wereinitially reviewed. Of these, 205 studies met thegeneral NRP methodological criteria and were thenclassified into instructional categories based on thekind of instruction used. Application of the morespecific review criteria precluded formal meta-analyses because of the large variation inmethodologies and implementations used. The Panelfound few research studies that met all NRP researchmethodology criteria. Nevertheless, the Panelemployed the NRP criteria to the maximum extentpossible in its examination of this body of literature.(See the Comprehension section of the Report of theNational Reading Panel: Reports of the Subgroups.)

In its review, the Panel identified 16 categories of textcomprehension instruction of which 7 appear to havea solid scientific basis for concluding that these typesof instruction improve comprehension in non-impairedreaders. Some of these types of instruction are helpfulwhen used alone, but many are more effective whenused as part of a multiple-strategy method. The typesof instruction are:

Comprehension monitoring, where readers learnhow to be aware of their understanding of thematerial;

Cooperative learning, where students learnreading strategies together;

Use of graphic and semantic organizers (includingstory maps), where readers make graphicrepresentations of the material to assistcomprehension;

Question answering, where readers answerquestions posed by the teacher and receiveimmediate feedback;

Question generation, where readers askthemselves questions about various aspects of thestory;

Story structure, where students are taught to usethe structure of the story as a means of helpingthem recall story content in order to answerquestions about what they have read; and

Summarization, where readers are taught tointegrate ideas and generalize from the textinformation.

Findings and DeterminationsIn general, the evidence suggests that teaching acombination of reading comprehension techniques isthe most effective. When students use themappropriately, they assist in recall, question answering,question generation, and summarization of texts.When used in combination, these techniques canimprove results in standardized comprehension tests.

Nevertheless, some questions remain unanswered.More information is needed on ways to teach teachershow to use such proven comprehension strategies.The literature also suggests that teachingcomprehension in the context of specific academicareasfor example, social studiescan be effective.If this is true of other subject areas, then it might beefficient to teach comprehension as a skill in contentareas.

Questions remain as to which strategies are mosteffective for which age groups. More research isnecessary to determine whether the techniques applyto all types of text genres, including narrative andexpository texts, and whether the level of difficulty ofthe texts has an impact on the effectiveness of thestrategies. Finally, it is critically important to knowwhat teacher characteristics influence successfulinstruction of reading comprehension.

Teacher Preparation and ComprehensionStrategies Instruction

Teaching reading comprehension strategies to studentsat all grade levels is complex. Teachers not only musthave a firm grasp of the content presented in text, butalso must have substantial knowledge of the strategies

159 1--)a.

Report of the National Reading Panel


themselves, of which strategies are most effective fordifferent students and types of content and of howbest to teach and model strategy use.

Research on comprehension strategies has evolveddramatically over the last 2 decades. Initially,investigators focused on teaching one strategy at atime; later studies examined the effectiveness ofteaching several strategies in combination. However,implementation of this promising approach has beenproblematic. Teachers must be skillful in theirinstruction and be able to respond flexibly andopportunistically to students' needs for instructivefeedback as they read.

The initial NRP search for studies relevant to thepreparation of teachers for comprehension strategyinstruction provided 635 citations. Of these, only fourstudies met the NRP research methodology criteria.Hence, the number of studies eligible for furtheranalysis precluded meta-analysis of the data derivedfrom these investigations. However, because therewere only four studies, the NRP was able to reviewthem in detail. The studies investigate two majorapproaches: Direct Explanation and TransactionalStrategy Instruction.

The Direct Explanation approach focuses on theteacher's ability to explain explicitly the reasoning andmental processes involved in successful readingcomprehension. Rather than teach specific strategies,teachers help students (1) to view reading as aproblem solving task that necessitates the use ofstrategic thinking, and (2) to learn to think strategicallyabout solving comprehension problems. For example,teachers are taught that they could teach students theskill of finding the main idea by casting it as aproblemsolving task and reasoning about itstrategically.

Transactional Strategy Instruction also emphasizes theteacher's ability to provide explicit explanations ofthinking processes. Further, it emphasizes the abilityof teachers to facilitate student discussions in which

students collaborate to form joint interpretations oftext and acquire a deeper understanding of the mentaland cognitive processes involved in comprehension.

Findings and DeterminationsThe four studies (two studies for each approach)demonstrated that teachers could be instructed inthese methods. Teachers required instruction inexplaining what they are teaching, modeling theirthinking processes, encouraging student inquiry, andkeeping students engaged. Data from all four studiesindicated clearly that in order for teachers to usestrategies effectively, extensive formal instruction inreading comprehension is necessary, preferablybeginning as early as preservice.

More research is needed to address the followingquestions. Which components of teacher preparationare most effective? Can reading comprehensionstrategies be successfully incorporated into contentarea instruction? How can the effectiveness ofstrategies be measured in an optimal manner? Canstrategies be taught as early as grades 1 and 2, whenchildren also are trying to master phonics, wordrecognition, and fluency? How can teachers be taughtto provide the most optimal instruction?

Teacher Education and ReadingInstructionRecent developments such as class size reduction andthe writing of standards suggest the growingimportance of teacher education on learning outcomes.In addition, the National Reading Panel de'cided tofocus on this area because during its regional meetingsspeakers expressed intense interest in the quality andimportance of teacher education.

In teacher education programs, preservice teachersgenerally acquire knowledge through supervisedteaching and through coursework in theory andmethods. Continuing education for practicing teacherscomes from professional development, also calledinservice education. The NRP analysis on this topicwas guided by three primary questions: How areteachers taught to teach reading? What does research



show about the effectiveness of this instruction? Howcan research be applied to improve teacherdevelopment? The initial literature search by the Panelidentified more than 300 articles. A total of 32 studiesmet the methodological NRP criteria: 11 preserviceand 21 inservice. No meta-analysis was conductedbecause the range of variables and theoreticalpositions was too large for the limited number ofstudies.

Findings and DeterminationsAs indicated by the NRP's examination of theliterature, only a small number of experimental studieshave been published about the effectiveness ofpreservice and inservice teacher education. Forconclusions to be drawn about the effectiveness ofteacher education, information on both teacher andstudent outcomes must be reported. Preserviceresearch, however, only measured teacher outcomes,whereas ideally both short- and long-term teacher andstudent outcomes should be observed. With respectto research on inservice education, only about one-halfmeasured student outcomes as well as teacheroutcomes.

Generally the results indicated that inserviceprofessional development produced significantly higherstudent achievement. There were few studies of thelong-term maintenance of the gains. While there wereonly a small number of studies, almost all of themshowed positive effects on teaching. However, therewere too few studies on specific variables to allow thePanel to draw specific conclusions about the contentof preservice education.

More information is needed in several areas. What isthe optimal combination of preservice and inserviceeducation, and what are the effects of preserviceexperience on inservice performance? What is theappropriate length of inservice and preserviceeducation? What are the best ways to assess theeffectiveness of teacher education and professional

development? How can teachers optimally besupported over the long term to ensure sustainedimplementation of new methods and to ensure studentachievement? The relationship between thedevelopment of standards and teacher education isalso an important gap in current knowledge.

Computer Technology andReading InstructionUntil recently, computers were not considered capableof delivering reading instruction effectively. They couldnot comprehend oral reading and judge its accuracy.They also were unable to accept free-form responsesto comprehension questions, so their use had to relyprimarily on multiple-choice formats. Today, thesituation is much improved. New computers havespeech recognition capabilities as well as manymultimedia presentation functions. Developments inthe Internet, with possibilities of linking schools andinstruction, have further increased interest intechnology as a teaching device. Computertechnology is different from other areas the NRPanalyzed. It cannot be studied independently ofinstructional content and is not an instructional methodin itself. Thus, computer technology must be examinedfor its ability to deliver instruction, for example, invocabulary or in phonemic awareness.

Because this is a relatively new field, the number ofstudies published in this area is small. Only 21 studiesmet the NRP research methodology criteria.

Findings and DeterminationsAlthough it is difficult to draw conclusions from thesestudies, it is possible to make some generalstatements. First, all the studies report positive results,suggesting that it is possible to use computertechnology for reading instruction. The seven studiesthat reviewed the addition of speech to computer-presented text indicate that this may be a promisinguse of technology in reading instruction.



Two other trends show promise. The use of hypertext(highlighted text that links to underlying definitions orsupporting or related text, almost like an electronicfootnote), while technically not reading instruction,may have an instructional advantage. Second, the useof computers as word processors may be very useful,given that reading instruction is most effective whencombined with writing instruction.

Striking in its absence is research on the incorporationof Internet applications to reading instruction.

Research also is needed on the value of speechrecognition as a technology and the use of multimediapresentations in reading instruction.

In sum, the Panel is encouraged by the reportedsuccesses in the use of computer technology forreading instruction, but relatively few specificinstructional applications can be gleaned from theresearch. Many questions still need to be addressed.


Next Steps

As part of its Congressional charge, the NRP wasdirected to assess the effectiveness of variousapproaches to teaching children to read and to furtherindicate the extent to which effective approaches wereready for application in classroom settings. Theinstructional topics of alphabetics (phonemicawareness and phonics), fluency, comprehension(vocabulary instruction and text comprehension),teacher education and reading instruction, andcomputer technology and reading instructionaddressed in this Report were selected by the Panelfrom a candidate list of 35 topics generated fromPanel members' own expertise, from the report of theNational Research Council on Preventing ReadingDifficulties in Young Children (Snow, Bums, & Griffin,1998), and from the input the Panel received in itsregional hearings. Several additional factorscontributed to the consensus decision to limit thenumber of topics that could be addressed and toevaluate the research literature relevant to thesespecific topics. These factors included (1) thehypothesized role that these topics play in readinginstruction; (2) the availability of well-designedexperimental or quasi-experimental studies ofinstructional effectiveness for each of these topic areasversus other topic areas; (3) the immensity of theresearch literature in reading development and readinginstruction; and (4) constraints on time and Panelresources.

The Panel regrets that it could not evaluate all of thereading instructional topics that were identified byPanel members as well as by parents, educators, andpolicymakers at the regional meetings. The Panelemphasizes that omissions of topics such as the effectsof predictable and decodable text formats onbeginning reading development, motivational factors inlearning to read, and the effects of integrating readingand writing, to name a few, are not to be interpretedas determinations of unimportance or ineffectiveness.Indeed, each of the reports of the subgroups identifiesareas for future research. These can serve as

checklists of important research opportunities forfurther analyses and evaluations of the kind conductedby the Panel on this first set of topics.

It is the Panel's fervent hope that future evaluations ofimportant reading research topics will include ananalysis and assessment of correlational, descriptive,and qualitative studies that inform our understanding ofthe developmental reading process, and adetermination of what instructional implications can bedrawn from them. Moreover, it will be critical tounderstand better how quantitative, hypothesis-drivenstudies can best be integrated with qualitativeapproaches to obtain maximum reliability andecological validity. Likewise, it will be critical toidentify the most important methodological featuresinherent in qualitative and descriptive researchapproaches that lead to the collection of trustworthyevidence. Thus, the Panel recommends that theevaluation of these types of qualitative researchapproaches, methods, and evidence be guided by thedevelopment of a comparable methodologicallyrigorous review process similar to that employed bythe NRP with procedures and criteria designated apriori and applied within an open and public forum.

With this information as background, it is clear to thePanel that at least four major tasks remain indeveloping a science of reading development andreading instruction. First, where possible, there shouldbe meta-analyses of existing experimental or quasi-experimental research in topic areas not addressed bythe NRP. Second, additional experimental researchshould be conducted on questions unanswered by thePanel's analyses of the topics it did cover. Third, thereshould be an exhaustive and objective analysis ofcorrelational, descriptive, and qualitative studiesrelevant to reading development and readinginstruction that is carried out with methodological rigorfollowing pre-established criteria. Fourth,experimental research should be initiated to test those



hypotheses derived from existing correlational,descriptive, and qualitative research meeting highmethodological standards.

Following are three illustrative examples of importantreading research opportunities.

Student Populations. An important question iswhether students with learning disabilities havedistinctive instructional needs and whether theybenefit from instructional techniques that aredifferent from those that are optimal for other low-achieving (non-disabled) students. The Panel wasable to address this question with respect tophonemic awareness and phonics instructionalprograms and techniques. It found that both typesof students benefit from similar phonemicawareness and phonics instructional programs andtechniques. Because of the limited amount ofresearch available, the Panel could not answer thisquestion with respect to instructional programs andtechniques aimed at developing reading fluency

and comprehension. These importantcomparisons should be the focus of futureresearch.

Teacher Education. The primary purpose ofteacher education research is to inform theeffective practice of classroom teachers in order toimprove student performance. Rigorousexperimental and qualitative research that definesand characterizes effective teaching methodologiesthat demonstrate improved student performance islimited. This persistent and major gap in the extantknowledge base must be addressed. Effortsshould be made to answer the important questionsin this critical area.

Uses of Technology in Teaching Reading. Here .

again, credible experimental and qualitativeresearch is lacking. This is understandable in lightof the recent development of the relevanttechnology and its application to readinginstruction and student learning. Nevertheless, thePanel believes that this is an important andessentially unexplored field.


Reflections

The findings and determinations of the NRP reflect afocused and persistent effort on the part of the Panelto contribute reliable, valid, and trustworthyinformation to the body of knowledge that is leadingto a better scientific understanding of readingdevelopment and reading instruction. In carrying outits Congressional charge, the Panel was able to firstdevelop, and then to apply a methodologicallyrigorous research review process and protocol and todo so within an open and public forum. The a prioriestablishment of research review criteria, thesystematic evaluation process, and the openness topublic scrutiny at all times ensured that the evidenceultimately evaluated by the Panel met well-establishedobjective scientific standards. This process alsoserves as a model for future evaluations of evidenceobtained experimentally on other topics relevant toreading as well as for studies employingnonexperimental methodologies.

The work of the NRP builds on existing knowledgeabout what types of skills children need to acquire tobecome independent readers. Specifically, the Paneladdresses the evidence about what those skills are and

adds further knowledge about how those skills arebest taught to beginning readers who vary in initialreading-related abilities. The Panel identified anumber of instructional approaches, methods, andstrategies that hold substantial promise for applicationin the classroom at this time. Specifically, the Reportof the National Reading Panel: Reports of theSubgroups includes specific findings that can be usefulin helping teachers develop instructional applicationswith students. Moreover, the Reports of theSubgroups provides extensive references that teacherscan locate for instructional ideas and guidance. Inaddition, the Panel identified areas where significantlygreater research effort is needed, and where thequality of the research efforts must improve in order todetermine objectively the effectiveness of differenttypes of reading instruction. Significantly, the Panelhas reached a series of positive conclusions aboutseveral areas of instructional research through arigorous and open process. We are confident that thedeterminations made by the Panel in this regard willbenefit children, teachers, and educationalpolicymakers.

21

26Report of the National Reading Panel

References

Cooper, H., & Hedges, L.V. (1994). The handbookof research synthesis. New York: Russell SageFoundation.

Durkin, D. (1993). Teaching them to read (6th ed.).Boston, MA: Allyn & Bacon.

Harris, T., & Hodges, R. (Eds.). (1995). Theliteracy dictionary (p. 207). Newark, DE: InternationalReading Association.

Snow, C. E., Burns, S. M., & Griffin, P. (Eds.).(1998). Preventing reading difficulties in young children.Washington, DC: National Academy Press.

Whipple, G (Ed.). (1925). The Twenty-fourthYearbook of the National Society for the Study ofEducation: Report of the National Committee onReading. Bloomington, IL: Public School PublishingCompany.

23


ADDENDUM

28

Methodology: Processes Applied to the Selection, Review,and Analysis of Research Relevant to Reading Instruction

In an important action critical to its Congressionalcharge, the NRP elected to develop and adopt a setof rigorous research methodological standards. Thesestandards, which are defined in this section, guided thescreening of the research literature relevant to eachtopic area addressed by the Panel. This screeningprocess identified a final set of experimental or quasi-experimental research studies that were then subjectedto detailed analysis. The evidence-basedmethodological standards adopted by the Panel areessentially those normally used in research studies ofthe efficacy of interventions in psychological andmedical research. These include behaviorally basedinterventions, medications, or medical proceduresproposed for use in the fostering of robust health andpsychological development and the prevention ortreatment of disease. It is the view of the Panel thatthe efficacy of materials and methodologies used in theteaching of reading and in the prevention or treatmentof reading disabilities should be tested no lessrigorously. However, such standards have not beenuniversally accepted or used in reading educationresearch. Unfortunately, only a small fraction of thetotal reading research literature met the Panel'sstandards for use in the topic analyses.

With this as background, the Panel understood thatcriteria had to be developed as it considered whichresearch studies would be eligible for assessment.There were two reasons for determining suchguidelines or rules a priori. First, the use of commonsearch, selection, analysis, and reporting procedureswould ensure that the Panel's efforts could proceed,not as a diverse collection of independentandpossibly unevensynthesis papers, but as parts of agreater whole. The use of common procedurespermitted a more unified presentation of the combinedmethods and findings. Second, the amount ofresearch synthesis that had to be accomplished wassubstantial. Consequently, the Panel had to work in

diverse subgroups to identify, screen, and evaluate therelevant research to complete their respective reports.Moreover, the Panel also had to arrive at findings thatall or nearly all of the members of the NRP couldendorse. Common procedures, grounded in scientificprinciples, helped the Panel to reach final agreements.

Search ProceduresEach subgroup conducted a search of the literatureusing common procedures, describing in detail thebasis and rationale for its topical term selections, thestrategies employed for combining terms or delimitingsearches, and the search procedures used for eachtopical area.

Each subgroup limited the period of time covered byits searches on the basis of relative recentness andhow much literature the search generated. Forexample, in some cases it was decided to limit theyears searched to the number of most recent yearsthat would identify between 300 and 400 potentialsources. This scope could be expanded in lateriterations if it appeared that the nature of the researchhad changed qualitatively over time, if the proportionof useable research identified was small (e.g., less than25%), or if the search simply represented too limited aproportion of the total set of identifiable studies.Although the number of years searched varied amongsubgroup topics, decisions regarding the number ofyears to be searched were made in accord withshared criteria.

The initial criteria were established to focus the effortsof the Panel. First, any study selected had to focusdirectly on children's reading development frompreschool through grade 12. Second, the study had tobe published in English in a refereed journal. At aminimum, each subgroup searched both PsyclNFOand ERIC databases for studies meeting these initialcriteria. Subgroups could, and did, use additional

27



databases when appropriate. Although the use of aminimum of two databases identified duplicateliterature, it also afforded the opportunity to expandperspective and locate articles that would not beidentifiable through a single database.

Identification of each study selected was documentedfor the record, and each was assigned to one or moremembers of the subgroup, who examined the title andabstract. Based on this examination, the subgroupmember(s) determined, if possible at this stage,whether the study addressed issues within the purviewof the research questions being investigated. If it didnot, the study was excluded and the reason(s) for theexclusion were detailed and documented for therecord. If it did address reading instructional issuesrelevant to the Panel's selected topic areas, the studyunderwent further examination.

Following initial examination, if the study had not beenexcluded in accord with the preceding criteria, the fullstudy report was located and examined in detail todetermine whether the following criteria were met:

Study participants must be carefully described(age, demographic, cognitive, academic, andbehavioral characteristics);

Study interventions must be described in sufficientdetail to allow for replicability, including how longthe interventions lasted and how long the effectslasted;

Study methods must allow judgments about howinstruction fidelity was insured; and

Studies must include a full description of outcomemeasures.

These criteria for evaluating research literature arewidely accepted by scientists in disciplines involved inmedical, behavioral, and social research. Theapplication of these criteria increases the probabilitythat objective, rigorous standards were used and thattherefore the information obtained from the studieswould contribute to the validity of any conclusionsdrawn.

If a study did not meet these criteria or could not belocated, it was excluded from subgroup analysis andthe reason(s) for its exclusion detailed anddocumented for the record. If the study was locatedand met the criteria, the study became one of thesubgroup's core working set of studies. The coreworking sets of studies gathered by the subgroupswere then coded as described below and thenanalyzed to address the questions posed in theintroduction and in the charge to the Panel.

If a core set of studies identified by the subgroup wasinsufficient to answer critical instructional questions,less recent studies were screened for eligibility for, andinclusion in, the core working sets of studies. Thissecond search used the reference lists of all corestudies and known literature reviews. This processidentified cited studies that could meet the Panel'smethodological criteria for inclusion in the subgroups'core working sets of studies. Any second search wasdescribed in detail and applied precisely the samesearch, selection, exclusion, and inclusion criteria anddocumentation requirements as were applied in thesubgroups' initial searches.

Manual searches, again applying precisely the samesearch, selection, exclusion, and inclusion criteria anddocumentation requirements as were applied in thesubgroups' electronic searches, were also conductedto supplement the electronic database searches.Manual searching of recent journals that publishresearch on specific NRP subgroup topics wasperformed to compensate for the delay in appearanceof these journal articles in the electronic databases.Other manual searching was carried out in relevantjournals to include eligible articles that should havebeen selected, but were missed in electronic searches.

Source of Publications: The Issue ofRefereed and Non-RefereedArticlesThe subgroup searches focused exclusively onresearch that had been published or had beenscheduled for publication in refereed (peer-reviewed)journals. The Panel reached consensus that



determinations and findings for claims and assumptionsguiding instructional practice depended on suchstudies. Any search or review of studies that had notbeen published through the peer review process butwas consulted in any subgroup's review was treatedas separate and distinct from evidence drawn frompeer reviewed sources (i.e., in an appendix) and is notreferenced in the Panel's report. These non-peer-reviewed data were treated as preliminary/pilot datathat might illuminate potential trends and areas forfuture research. Information derived in whole or inpart from such studies was not to be represented atthe same level of certainty as findings derived from theanalysis of refereed articles.

Types of Research Evidence andBreadth of Research MethodsConsideredDifferent types of research (e.g., descriptive-interpretive, correlational, experimental) lay claim toparticular warrants, and these warrants differmarkedly. The Panel felt that it was important to use awide range of research, but that the research be usedin accordance with the purposes and limitations of thevarious research types.

To make a determination that any instructional practicecould be or should be adopted widely to improvereading achievement requires that the belief,assumption, or claim supporting the practice is causallylinked to a particular outcome. The highest standardof evidence for such a claim is the experimental study,in which it is shown that treatment can make suchchanges and effect such outcomes. Sometimes whenit is not feasible to do a randomized experiment, aquasi-experimental study is conducted. This type ofstudy provides a standard of evidence that, while notas high, is acceptable, depending on the study design.

To sustain a claim of effectiveness, the Panel felt itnecessary that there be experimental or quasi-experimental studies of sufficient size or number, andscope (in terms of population served), and that these

studies be of moderate to high quality. When therewere too few studies of this type or they were toonarrowly cast or they were of marginally acceptablequality, then it was essential that the Panel havesubstantial correlational or descriptive studies thatconcurred with the findings if a claim was to besustained. No claim could be determined on the basisof descriptive or correlational research alone. The useof these procedures increased the possibility ofreporting findings with a high degree of internal validity.

Coding of DataCharacteristics and outcomes of each study that metthe screening criteria described above were codedand analyzed, unless otherwise authorized by thePanel. The data gathered in these coding forms werethe information submitted to the final analyses. Thecoding was carried out in a systematic and reliablemanner.

The various subgroups relied on a common codingform developed by a working group of the Panel'sscientist members and modified and endorsed by thePanel. However, some changes could be made to thecommon form by the various subgroups for addressingdifferent research issues. As coding forms weredeveloped, any changes to the common coding formwere shared with and approved by the Panel to ensureconsistency across various subgroups.

Unless specifically identified and substantiated asunnecessary or inappropriate by a subgroup andagreed to by the Panel, each form for analyzing studieswas coded for the following categories:

1. Reference

Citation (standard APA format)

How this paper was found (e.g., search of nameddatabase, listed as reference in another empiricalpaper or review paper, manual search of recentissues of journals)

Narrative summary that includes distinguishingfeatures of this study


31


2. Research Question: The generalumbrella question that this studyaddresses

3. Sample of Student Participants

States or countries represented in sample

Number of different schools represented in sample

Number of different classrooms represented insample

Number of participants (total, per group)

Age

Grade

Reading levels of participants (prereading,beginning, intermediate, advanced)

Whether participants were drawn from urban,suburban,.or rural settings

List any pretests that were administered prior totreatment

List any special characteristics of participantsincluding the following if relevant:

Socioeconomic status (SES)

Ethnicity

Exceptional learning characteristics, such as:

Learning disabled

- Reading disabled

Hearing impaired

English language learners (ELL); also known aslimited English proficient (LEP) students

Explain any selection restrictions that were appliedto limit the sample of participants (e.g., only thoselow in phonemic awareness were included)

Contextual information: concurrent readinginstruction that participants received in theirclassrooms during the study

Was the classroom curriculum described in thestudy? (code = yes/no)

Describe the curriculum

Describe how sample was obtained:

Schools or classrooms or students were

selected from the population of those available

Convenience or purposive sample

Not reported

Sample was obtained from another study(specify study)

Attrition:

Number of participants lost per group duringthe study

Was attrition greater for some groups than forothers? (yes/no)

4 Setting of the Study

5.

Classroom

Laboratory

Clinic

Pullout program (e.g., Reading Recovery©)

Tutorial

Design of Study

Random assignment of participants to treatments(randomized experiment)

- With vs. without a pretest

Nonequivalent control group design (quasi-experiment), e.g., existing groups assigned totreatment or control conditions, no randomassignment

With vs. without matching or statistical controlto address nonequivalence issue

One-group repeated measure design (i.e., onegroup receives multiple treatments, considered aquasi-experiment)

Treatment components administered in a fixedorder vs. order counterbalanced acrosssubgroups of participants

Multiple baseline (quasi-experiment)

- Single-subject design

Aggregated-subjects design

6. Independent Variables

Treatment Variablesa.



Describe all treatments and control conditions; besure to describe nature and components of readinginstruction provided to control group.

For each treatment, indicate whether instructionwas explicitly or implicitly delivered and, if explicitinstruction, specify the unit of analysis (sound-symbol; onset/rime; whole word) or specificresponses taught. [Note: If this category isomitted in the coding of data, justification must beprovided.]

If text is involved in treatments, indicate difficultylevel and nature of texts used

Duration of treatments (given to students)

Minutes per session

Sessions per week

Number of weeks

Was trainers' fidelity in delivering treatmentchecked? (yes/no)

Properties of teachers/trainers

Number of trainers who administered treatments

Teacher/student ratio: Number of trainers tonumber of participants

Type of trainer (classroom teacher, studentteacher, researcher, clinician, special educationteacher, parent, peer, other)

List any special qualifications of trainers

Length of training given to trainers

Source of training

Assignment of trainers to groups:

Random

Choice/preference of trainer

- All trainers taught all conditions

Cost factors: List any features of the training suchas special materials or staff development oroutside consultants that represent potential costs

b. Moderator VariablesList and describe other nontreatmentindependent variables included in the analysesof effects (e.g., attributes of participants,properties or types of text)

7. Dependent (Outcome) Variables

List processes that were taught during training andmeasured during and at the end of training

List names of reading outcomes measured

Code each as standardized or investigator-constructed measure

Code each as quantitative or qualitativemeasure

For each, is there any reason to suspect lowreliability? (yes/no)

List time points when dependent measures wereassessed

8. Nonequivalence of groups

Any reason to believe that treatment/control groupmight not have been equivalent prior totreatments? (yes/no)

Were steps taken in statistical analyses to adjustfor any lack of equivalence? (yes/no)

9. Result (for each measure)

Record the name of the measure

Record whether the differencetreatment meanminus control meanis positive or negative

Record the value of the effect size including its sign(+ or -)

Record the type of summary statistics from whichthe effect size was derived

Record number of people providing the effect sizeinformation

10. Coding information

Record length of time to code study

Record name of coder

If text was a variable, the coding indicated what isknown about the difficulty level and nature of the textsbeing used. Any use of special personnel to deliver anintervention, use of special materials, staffdevelopment, or other features of the intervention thatrepresent potential cost were noted. Finally, variousthreats to reliability and internal or external validity

31 Report of the National Reading Panel3 3


(group assignment, teacher assignment, fidelity oftreatment, and confounding variables includingequivalency of subjects prior to treatment anddifferential attrition) were coded. Each subgroup alsocoded additional items deemed appropriate orvaluable to the specific question being studied by thesubgroup members.

A study could be excluded at the coding stage only if itwas found to have so serious a fundamental flaw thatits use would be misleading. The reason(s) forexclusion of any such study was detailed anddocumented for the record. When quasi-experimentalstudies were selected, it was essential that each studyincluded both pre-treatment and post-treatmentevaluations of performance and that there was acomparison group or condition.

Each subgroup conducted an independent re-analysisof a randomly designated 10% sample of studies.Absolute rating agreement was calculated for eachcategory (not for forms). If absolute agreement fellbelow 0.90 for any category for occurrence ornonoccurrence agreement, the subgroup took someaction to improve agreement (e.g., multiple readingswith resolution, improvements in coding sheet).

Upon completion of the coding for recently publishedstudies, a letter was sent to the first author of the studyrequesting any missing information. Any informationthat was provided by authors was added to thedatabase.

After its search, screening, and coding, a subgroupdetermined whether for a particular question or issue ameaningful meta-analysis could be completed orwhether it was more appropriate to conduct aliterature analysis of that issue or question withoutmeta-analysis, incorporating all of the informationgained. The full Panel reviewed and approved ormodified each decision.

Data AnalysisWhen appropriate and feasible, effect sizes werecalculated for each intervention or condition inexperimental and quasi-experimental studies. Thesubgroups used the standardized mean differenceformula as the measure of treatment effect. Theformula was:

(Mt Mc) / 0.5(sdt + sdc)

where:

M is the mean of the treated group,

Mc is the mean of the control group,

sdt is the standard deviation of the treated group, and

sde is the standard deviation of the control group.

When means and standard deviations were notavailable, the subgroups followed the guidelines for thecalculation of effect sizes as specified by Cooper andHedges (1994).

The subgroups weighted effect sizes by numbers ofsubjects in the study or comparison to prevent smallstudies from overwhelming the effects evident in largestudies.

Each subgroup used median and/or average effectsizes when a study had multiple comparisons, andeach subgroup only employed the comparisons thatwere specifically relevant to the questions underreview by the subgroup.

Expected OutcomesAnalyses of effect sizes were undertaken with severalgoals in mind. First, overall effect sizes of relatedstudies were calculated across subgroups to determinethe best estimate of a treatment's impact on reading.These overall effects were examined with regard totheir difference from zero (i.e., does the treatmenthave an effect on reading?), strength (i.e., if thetreatment has an effect, how large is that effect?), andconsistency (i.e., did the effect of the treatment vary


An Evidence-Based Assessment 0significantly from study to study?). Second, the Panelcompared the magnitude of a treatment's effect underdifferent methodological conditions, program contexts,program features, and outcome measures and forstudents with different characteristics. Theappropriate moderators of a treatment's impact weredrawn from the distinctions in studies recorded on thecoding sheets. In each case, a statistical comparisonwas made to examine the impact of each moderatorvariable on average effect sizes for each relevantoutcome variable. These analyses enabled the Panelto determine the conditions that alter a program'seffects and the types of individuals for whom the

program is most and least effective. Within-groupaverage effect sizes were examined as were overalleffect sizes for differences from zero and for strength.The analytic procedures were carried out using thetechniques described by Cooper and Hedges (1994).

ReferencesCooper, H., & Hedges, L. V. (1994). The handbook

of research synthesis. New York: Russell SageFoundation.

Snow, C. E., Burns, S. M., & Griffin, P. (Eds.).(1998). Preventing reading difficulties in young children.Washington, DC: National Academy Press.

33 r Report of the National Reading Panel

U.S. Department of Health and Human Services

Public Health Service

National Institutes of Health

National Institute of Child Health and Human Development

NIH Pub. No. 00-4769

April 2000

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

ERIC

This document is covered by a signed "Reproduction Release(Blanket) form (on file within the ERIC system), encompassing all

or classes of documents from its source organization and, therefore,

does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission to

reproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Release form

(either "Specific Document" or "Blanket").

EFF-089 (9/97)

reproductions supplied by edrs are the best that can be made · blair johnson alisa kenny helen s....

Documents