instructional program coherence: what it is and why it ... · instructional program coherence...

Educational Evaluation and Policy AnalysisWinter 2001, Vol. 23, No. 4, pp. 297-321

Instructional Program Coherence: What It Isand Why It Should Guide School Improvement Policy

Fred M. NewmannUniversity of Wisconsin, Madison

BetsAnn SmithMichigan State University

Elaine AllensworthConsortium on Chicago School Research

Anthony S. BrykUniversity of Chicago

We present the concept of instructional program coherence and explain why school improvementframeworks that incorporate instructional program coherence are more likely to advance studentachievement than multiple, unrelated efforts. We present evidence that Chicago elementary schoolswith stronger instructional program coherence make higher gains in student achievement. We alsoshare observations on how, in specific schools, principals and external partners directed key schoolresources toward the development of instructional program coherence. In closing, we discuss factorswithin the educational system that discourage instructional program coherence and suggest ways thatschool leaders, school improvement partners, and policymakers can support greater instructionalprogram coherence.

In this article, we propose that reform effortsmay fail to improve student achievement if theyfail to strengthen instructional program coher-ence within schools. We define instructional pro-gram coherence as a set of interrelated programsfor students and staff that are guided by a com-mon framework for curriculum, instruction, as-sessment, and learning climate and that are pursuedover a sustained period. Reforn that strengthensinstructional program coherence contrasts withefforts to improve schools through the adoptionof a wide variety of programs that are often un-coordinated or limited in scope or duration. Weexplain why such approaches, however innovative,are less likely to advance student achievement thanare more coherent programs. We test and supportour argument with data from Chicago elementaryschools that show greater achievement gains in

schools with more coherent instructional pro-grams. Using field study data-from a set of schools,we then describe how some principals and exter-nal partners directed resources to develop greaterinstructional program coherence. 'In closing, wediscuss factors within the U.S. educational systemthat undermine instructional program coherence.We recommend that school leaders, school assis-tance organizations, and policymakers give moreattention to strengthening instructional programcoherence.

The Problem: Too Many Unrelated,Unsustained "Improvement" Programs

Over the past decade, many poorly performingelementary schools have sought to improve theirinstructional programs and outcomes by adoptingnumerous school improvement projects, programs,

297

Newmann, Smith, Allensworth, and Bryk

and partnerships. The following pattern oftenemerges: Staffs divide themselves among vari-ous initiatives and direct a great deal of time andenergy into multiple workshops, meetings, andconferences. Immediate success may not be ex-pected from these efforts, but with time, desiredimprovements in student achievement gains failto materialize and professional fatigue and frus-tration rise. Many of these improvement programsfade or end, while new programs continue to beadopted.

These schools are caught in a bind. They wantto acquire programs and materials that mighthelp them to teach more effectively, but they soonfind themselves in a large and fragmented cir-cuit of school improvement activity. Principalsmay recognize that faculty members' attentionis scattered, but hooking up with multiple ini-tiatives seems to be the only way to gain neededresources and to promote the commitment ofstaff with different interests and strengths. More-over, the emotional and social needs of manystudents require external partnerships. With somany demands, principals feel unable to refuseprograms and reason that diverse programs willsomehow complement one another. They con-tinue to adopt or pilot programs but do little toestablish or strengthen coordination and coher-ence among them.

Prior Literature andDiscussion of Coherence

The idea of coherence often surfaces wheneducational researchers confront practices, pro-grams, or policies found to be poorly conceivedand coordinated or at odds with other practices,programs, and policies. To our knowledge, how-ever, no studies have offered a thoughtful, sys-tematic definition and exploration of school-levelinstructional program coherence. Some direct andindirect arguments in favor of greater curricular,organizational, or policy coherence in educationhave been raised, but they do not address threeimportant matters. First, they do not address in-structional program coherence and how it mightconstitute an important school improvement strat-egy. Second, they do not provide a theoretical ex-planation for why strong instructional programcoherence might be expected to advance studentlearning. And third, they do not offer an opera-tional definition that could support empirical studyof the effects of greater coherence on student learn-

298

ing. After reviewing how prior literature has mostoften discussed coherence, we address these threeconcems.

The concept of coherence has surfaced perhapsmost explicitly in discussions of curriculum im-provement (Cohen & Ball, 1996; Smith, Smith, &Bryk, 1998). Calls for coherent curriculum arguefor sensible connections and coordination be-tween the topics that students study in each sub-ject within a grade and as they advance throughthe grades.

Instructional program coherence entrails cur-ricular coherence, but entails several other criteriaas well.

More indirect are discussions of coherence,or the lack thereof, in the school organization lit-erature. For example, prior studies have docu-mented the importance of organizational factorssuch as unity of purpose, a clear focus, and sharedvalues for student learning (Bryk, Lee, & Holland,1993; Coleman, Hoffer, & Kilgore, 1982; Hill &Celio, 1998; Sergiovanni, 1994). Other researchhas drawn attention to schools where diverse im-provement initiatives haveno apparent effect oncore achievement goals because they lack the sus-tained attention of the majority of staff (Alling-ton & Johnson, 1989; Cohen & Ball, 1996; El-more, 1996; Newmann & Wehlage, 1995; Smylie,Bilcer, Kochanek, Sconzert, Shipps, & Swyers,1998). Similarly, early research on school reformin Chicago focused attention on the problem of"Christmas tree" innovation, that is, improvementefforts that brought attention to a school throughprogram and equipment purchases but that failedto build its capacity to improve teaching andlearning (Bryk, Sebring, et al., 1998). Concernfor greater school-level coherence is suggested inmovements toward "whole-school" reform such asthose advanced by the New American Schools De-velopment Corporation (New American SchoolsDevelopment Corporation, 1991). These reformsseek to replace limited or piecemeal improvementefforts with school reform models that demandsimultaneous change in a set of fundamental or-ganizational supports or "commonplaces," such ascurriculum, student grouping, staff development,and school governance. This approach to schoolimprovement is increasingly common. It is, forexample, used in the Chicago Public Schools(CPS), which requires schools to organize schoolimprovement planning around five essential sup-ports: school leadership, parent and community

Instructional Program Coherence

involvement, professional community, a student-centered learning climate, and high-quality in-struction (Bryk, Lee, & Smith, 1990; Bryk, Se-bring, et al., 1998; Chicago Public Schools, 1995;Newmann & Wehlage, 1995). Although whole-school models may spur more extensive improve-ment or restructuring efforts, they may or may notaddress relationships between supports and theircollective link to student learning. Schools can de-velop supports in more or less coherent ways, andall of the supports could be present without stronginstructional program coherence.

Finally, studies on the broader educational sys-tem tends to discuss coherence as aligmnent ofFaschool's instructional programn with extemal po]i-cies and standards (Consortium for Policy Re-search in Education, 2000; Furhman, 1993; Smith& O'Day, 1991). Such studies point out how clut-tered and contradictory state and district policy en-vironments can fragment school developmentefforts (Cohen. 1995; O'Day, Goertz, & Floden,1995), but they do not address how administra-tors and teachers might develop greater within-school coherence.

A Definition of InstructionalProgram Coherence

The aforementioned studies and others onschool improvement certainly imply the need forinstructional program coherence, but they havenot provided an operational definition. Based onour own studies of school innovation in diversecontexts and a review of research on learning, mo-tivation, organizational productivity, and schooleffectiveness, we define instructional programcoherence as a set of interrelated programs forstudents and staff that are guided by a commonframework for curriculum, instruction, assess-ment, and learning climate and are pursued overa sustained period. Strong program coherence isevident when three major conditions prevail in aschool:

1. A common instructionalframework guidescurriculum, teaching, assessment, and learningclimate. The framework combines specific expec-tations for student learning, with specific strate-gies and materials to guide teaching and assess-ment. One example is the Cunningham approachto literacy, which provides a framework for cur-riculum, instructional strategies, and assessmentsand which specifies that learning activities be

organized into four areas: guided reading, self-selected reading, working with words, and writ-ing (Cunningham & Allington, 1999). Anotherexample is Success for All's approach to mathe-matics, which provides grade-aligned curricu-lum, makes learning expectations clear (for ex-ample, students must engage in mathematicalreasoning in situations outside school), and spec-ifies instructional strategies for particular typesof leaming, such as manipulatives to representconcepts or symbols and cooperative learningmethods for problem solving (Slavin, Madden,Dolan, & Wasik, 1996). Although instructionalframeworks are often developed by extemal pro-viders, schools may also develop their own. In ei-ther case, a common instructional frameworkwould mean the;following:

a. Curriculum, instructional strategies, andassessments of students are coordinated amongteachers within a grade level.

b. Curriculum and assessments of studentsproceed logically from one grade level to thenext and offer a progression of increasinglycomplex subject matter rather than repeatingrudimentary material previously taught.

c. Key student support programs, such astutoring, remedial instruction, parent educa-tion, and opportunities for parent involvementfocus consistently on the school's instructionalframework.2. Staff working conditions support implemen-

tation of the framework.a. Administrators and teachers expect one

another to implement the framework.b. Criteria for recruiting and hiring teach-

ers emphasize commnitment to and competencein executing the framework.

c. Teachers are evaluated and held account-able largely on the basis of how effectively theyuse the common instructional framework,

d. Professional development opportuni-ties for staff are focused on the common in-structional framework, and professional devel-opment on complex topics is pursued over asustained period. (For instance, after beingintroduced to a new teaching strategy, teachershave several opportunities to critically examineit, to implement it in their classrooms, and toreceive feedback from colleagues and outsideexperts. The new strategy becomes a focus forteacher discussion and reflection for severalmonths or even years.)

299


3. The school allocates resources such asfunding, materials, time, and staff assignments toadvance the school's common instructionalframe-work and to avoid diffuse, scattered improvementefforts, with the following results:

a. Curriculum and student assessments re-main stable over time.

b. Teachers' professional assignments arestable enough that teachers have sustained op-portunities to learn how to teach well in theirspecific roles.

Our definition emphasizes focused and strate-gic coordination of key school supports for in-struction (especially curriculum), instructionalstrategies, assessments, student support programs,teaching assigmnents, expectations for teachers'performance, and in-service professional devel-opment. In these matters it differs from whole-school reform or even fidelity to a reform programbecause a school could faithfully implement acomprehensive collection of programs that donothing to build instructional coherence in theschool. That is, in fact, what we have increasinglyobserved.

Our definition also distinguishes instructionalprogram coherence from policy alignment with theextemal environment. We recognize that within-school coherence can be profoundly affected byconditions and policies beyond the school, butalignment to external policies may promote,undermine, or have no effect on the degree of in-structional program coherence. For example,school aligmnent with the following policies mightexacerbate program incoherence: (a) The statemandates curriculum standards that call for stu-dent mastery of hundreds of discrete competen-cies with no common themes or skills connect-ing them, or (b) the district recommends that allelementary teachers participate in one-day intro-ductory workshops on portfolio assessment, class-room management, higher order thinking, guidedreading, and culturally responsive teaching. In con-trast, district policy could assist instructional pro-gram coherence by requiring elementary schoolsto offer literacy instruction based on nationallyrecognized models and by offering a three-yearsequence of professional development for schoolteams to help them gain mastery in an approach.Thus any single policy could promote instructionalprogram coherence to a greater or lesser degree.Similarly, calls for systemic reform (Consortium

300

for Policy Research in Education, 2000; Fuhrman,1993; Smith & O'Day, 1991) and greater atten-tion to the coordination of local, state, and fed-eral policies could conceivably assist instruc-tional program coherence within schools, butonly if each individual policy reinforced programcoherence.

Why Should Instructional ProgramCoherence Promote Student Achievement?

Theory and research in the fields of learn-ing, motivation, organizational productivity,and school effectiveness suggest that instruc-tional program coherence should assist studentachievement in two ways: by helping teachers towork more effectively on problems of school im-provement and by directly increasing studentengagement and learning.

Assisting Student Learning and Engagement

Research on leaming and cognition indicatesthat students of all ages are more likely to learnwhen their experiences connect with and build onone another (Greeno, Collins, & Resnick, 1996;Mayer & Wittrock, 1996; Bransford, Brown, &Cocking, 1999). To the extent that experiencesare disconnected, it is more difficult for stu-dents to incorporate new understandings into priorknowledge and to alter prior knowledge when nec-essary. Studies in cognitive science indicate thatlearning takes time and requires recurring oppor-tunities to practice and to apply knowledge andskills in new contexts. Material leamed throughshort-term exposure and only in reference to alimited context is less likely to be retained andtransferred to other settings.

In comparison to disconnected, short-term ex-periences, integrated experiences that are sustainedlong enough for successful completion providegreater clarity about what is required for masteryand how prior knowledge can be applied to futurequestions. Students learning to read, for example,are more likely to gain basic skills and the confi-dence to tackle more challenging tasks if they leamin settings where all of their teachers assist theirreading in a consistent manner. In contrast, whenthere is little connection among past, present andfuture reading activities and when experiences aretoo brief to allow for mastery, it is more difficultfor students to process the information.

Most of the research on the importance of con-nected leaming experiences and the application

Instrmctional Program Coherence

of ideas across multiple contexts comes fromstudies on instruction within classrooms. But itis reasonable to think that those theories and in-sights apply to learning that occurs across variousclassrooms and to learning that occurs as studentsmove from one grade to the next.

Instructional program coherence may also as-sist student motivation. Research on motivationsuggests that students are more likely to engagein the difficult work of learning when curricularexperiences within classes, among classes, andover time are connected to one another (Pittman,1998; Newmann, 1981). As explained, coherentinstruction develops competence more effectivelythan incoherent instruction. When children seethemselves developing competence, they are moremotivated to work, because fulfilling the basichuman need for mastery builds confidence thatexerting effort will bring success (Ames & Ames,1984; Blauner, 1964; Kanfer, 1990). In contrast,when faced with incoherent activities, studentsare more likely to feel that they are targets of ap-parently random events and that they have lessknowledge of what should be done to succeed.Such feelings reduce student engagement in thehard work that learning often requires. Thus in-coherent activities undemnine opportunities to gainmastery and the confidence that motivates furtherlearning.

Together these points suggest that where cur-riculum, instruction, and special programs arecoordinated one can expect enhanced studentachievement. More coherent experiences can pro-vide absorbing activities that increase students'motivation to engage in learning and offer im-proved opportunities for cognitive processing.

Assisting Teacher Effectiveness

Instructional program coherence might also beexpected to assist teacher learning and effective-ness. According to the research on learning andmotivation summarized earlier, we would expectthat teachers who participate in coherent profes-sional development experiences, as opposed toshort-term, unrelated activities, are more likely tolearn from those experiences and to integrate thatnew knowledge into their teaching. In addition, re-search on organizations and effective managementindicates that professionals who work togetheron integrated activities aimed at clear goals pro-duce higher quality goods and services (Lawler,1990; Wohlstetter, Smyer, & Mohnnan, 1994).

One reason for this is that coordination of activ-ity amplifies workers' access to and use of techni-cal resources and expertise. For example, if teach-ers within a grade level pool their knowledge onthe most effective ways to use cooperative learn-ing in the study of mathematical estimation, eachteacher has an opportunity to improve his or herskill in applying cooperative learnng. Another rea-son is that connecting the work of various teach-ers to common purposes and practices that arepursued over an extended period gives teachers'work more meaning, thereby increasing theirmotivation and commitment to reach goals. Incontrast, when teachers know from prior experi-ence that initiatives are introduced and abandonedbefore they are substantially incorporated, it makeslittle sense to ekpend much effort to change one'spractice.

These points on student learning and engage-ment and teacher effectiveness are consistent withresearch on organizational factors that promoteschool effectiveness, including research on totalquality management (Consortium on Produc-tivity in the Schools, 1995; Pallas & Neumann,1995), effective schools (Purkey & Smith, 1983),high-capacity schools (King & Newmann, 2000),Catholic high schools (Bryk, Lee, et al., 1993),and the differences between public and privateschools (Chubb &Moe, 1990). Such studies arguethe importance of a sustained organizational focus,staff agreement on clear and specific goals, morecommon academic expectations and curriculumfor students, teacher collaboration and collectiveresponsibility for meeting goals, and a consistentclimate of positive supports and high expectationsfor all students and staff. Each of these elements isreflected in one or more of the indicators that wepropose for instructional program coherence.

Study Context, Design, and Data

Our central empirical question was whetherelementary schools' with improving instruc-tional coherence showed improvements in stu-dent achievement over multiple years. RecentCPS policy initiatives, including high-stakestesting and placing of schools on probation, haveprompted elementary schools to try a variety ofapproaches to raise students' standardized testscores. In addition, external organizations haveinvolved schools in a variety of improvement in-terventions. The abundance of improvement ac-tivity in Chicago provided a fitting opportunity to

301


examine the extent to which schools channeledtheir efforts into coherent instructional programsand whether that, in turn, resulted in improvedstudent achievement. We undertook a combina-tion of statistical analyses and field studies to in-vestigate these questions.

In trying to assess the strength of instructionalprogram coherence within schools, it is unreason-able to expect that all elements of a school can befully coordinated and aligned, or that a school canbe so disorganized as to show no coherence at all.Because many factors contribute to instructionalprogram coherence, our judgments about schoolcoherence were made on a continuum from lowto high, depending on how many indicators weremet and to what degree.

Samples and Data Collection

In 1994 and 1997, surveys with items measur-ing instructional program coherence were distrib-uted to teachers in all Chicago public elementaryschools by the Consortium on Chicago SchoolResearch. We used data from the 222 elementaryschools that participated both years. The surveysinvolved 5,358 teachers in 1994 with valid re-sponses on the coherence items and 5,560 teach-ers, also with valid responses, in 1997, with an av-erage of 24 teachers per school in 1994 and 25teachers per school in 1997.

We used student achievement data from theIowa Tests of Basic Skills (ITBS) in reading andmathematics, which were given to all students ingrades 2 through 8 in the same 222 schools from1993 through 1997. The number of students whotook the tests varied between grade levels, years,and the subjects of reading and mathematics, butthe average number of students taking the tests inany year was 81,493, with an average of 367 stu-dents per school.

We conducted field studies of 11 elementaryschools representing a diversity of approaches toschool improvement among external partners par-ticipating in the Chicago Annenberg Challenge(CAC). The CAC was a large-scale, five-year ini-tiative supporting whole-school efforts to improvestudent learning, in part by addressing three crit-ical issues: school and teacher isolation, schoolsize and personalism, and time for learning andimprovement. None of the schools were specialtyschools or magnets. They were located in neigh-borhoods that reflected a range of socioeconomicresources, although most struggled against highrates of poverty and social stress. Six of the schools

302

enrolled primarily African American students,two enrolled primarily Hispanic students, andthree enrolled racially mixed student populationswith Hispanic majorities.

Each of the 11 schools was studied by a two-person research team during the 1996-1997school year. The baseline school visits involvedabout 50 hours of observation and interviewing.Researchers observed and recorded language artsand math lessons in two classrooms each in gradesthree, six and eight to create a pool of 12 observa-tions. They also sat in on grade and committeemeetings taking place on the days when they werein the school. All observed teachers were inter-viewed, as were school curriculum coordinators,improvement program leaders, and the schoolprincipal. We also interviewed local school coun-cil and union representatives. Interviewees wereasked about school-improvement goals, activi-ties, and priorities; the extent to which goals wereheld in common and agreed to by staff; and in-structional objectives and coordination. Inter-viewed teachers were asked additional questionsabout their instructional goals and practices. Wealso asked all,staff to discuss work norms, rela-tionships among teachers, staff development ef-forts, and school-community relations. In addi-tion to the baseline school visit, one researcherreturned to the school several times to collect aca-demic tasks and work samples2 from the observedteachers, an informal process that generally addedto the team's school knowledge. Finally, the teamassembled and analyzed key school documents,including the annual school improvement plan,reports related to their participation in the CAC,data on student achievement, and organizationalprofiles based on school survey data analyzedby the Consortium on Chicago School Research.Drawing on all their school visits and data, eachteam produced an extensive school report de-scribing and analyzing the school's priority im-provement goals and actions, governance and op-erations, learning climate, instructional programand practices, professional development and com-munity, and parent and community relations. Weused those reports to assess the extent of instruc-tional program coherence in each school.

Measures

Survey measures

Several items on the surveys were used to con-struct measures of teachers' perceptions of school

program coherence. Survey items from 1994 wereas follows:

To what extent do you agree or disagree witheach of the following statements (strongly dis-agree, disagree, agree, strongly agree)?

* You can see real continuity from one pro-gram to another in this school.

* Many special programs come and go in thisschool. [reverse scored]

* Once we start a new program, we follow upto make sure that it's working.

* We have so many differentprograms in thisschool that I can't keep track of them all [reversescored]

Additional items were included in the 1997measure of school coherence:

To what extent do you agree or disagree witheach of the following statements (strongly dis-agree, disagree, agree, strongly agree)?

* Curriculum, instruction, and learning ma-terials are well coordinated across the differentgrade levels at this school.

* There is consistency in curriculum, instruc-tion, and learning materials among teachers inthe same grade level at this school.

* Most changes introduced at this school havelittle relation to teachers' and students' real needsand interests.

* Most changes introduced at this school helppromote the school's goals for learning.

To what extent have coordination andfocus ofthe school's instructionalprogram changed in thepast two years at your school (worse, no change,better)?

To allow for a direct comparison of teachers'responses from 1994 to 1997, we mathematicallyequated common items in the two surveys byusing Rasch Rating Scale analysis (Wright &Masters, 1982). The equating process allowed usto develop measures that were equivalent for bothyears (i.e., a score of 5 represented the same levelof coherence in both 1994 and 1997). It also al-lowed us to discern whether responses to the ad-ditional items in 1997 followed a different patternthan did those in the 1994 measure (i.e., whetherthey tapped a different underlying construct).3

The new items did not perform differently fromthe common items, indicating that the measurewas not substantively different in 1997 and 1994.4The reliability of the 1994 measure was .64; the re-


liability of the 1997 measure was .82.5 The mea-sures were then fit to a 10-point scale. EmpiricalBayes estimates of average school coherencewere computed through hierarchical linearmodels to produce a measure of each school'slevel of instructional program coherence in1994 and in 1997 .6

We analyzed the relationship between the sur-vey measures of coherence and the observers' rat-ings based on the 13-point rubric in the 11 schoolsstudied. We found a correlation of .70 betweenthese measures. The correlation assured us thatour measures had tapped important organizationaldifferences among schools. Thus we had reason tothink that the organizational differences identifiedin the field studies would generalize more broadlyacross elementary.schools in the CPS.

Achievement measures

We measured academic achievement by usingthe ITBS in reading and in math. The variousforms and test levels of the exam taken by studentsin grades 2 through 8 from 1994 to 1997 wereequated through Item Response Theory. The re-sult was h score, in logits, for each student thatcould be compared to other students' scores re-gardless of the form or level of the test that theytook (Bryk, Thum, Easton, & Luppescu, 1998;Van der Linden & Hambleton, 1997). On average,students' test scores between 1993 and 1997 in-creased by .6 logits per year. Therefore, we con-sidered each .6 logit change as a change equivalentto one year of learning. This number is approxi-mate, however, as the average gain per yearvaries by grade level, with lower grades averaginggains of about .7-.8 logits per year and uppergrades about .5-.6 logits per year.

Qualitative measures

We developed a rubric that assessed the extentof instructional program coherence in a school ac-cording to 13 indicators capturing the major com-ponents of our definition. Each school was rated oneach indicator by one of this report's authors andby the lead researcher for the school on the basis ofsummaries of classroom observation and interviewdata contained in the school's field report as theprimary evidence for the rating.7 All indicatorswere scored on a 4-point scale (1 = not at all,4 = to a great extent). The specific criteria for eachindicator's rating are included in the appendix. Onaverage, 93% of the initial ratings from a schoolwere either in precise agreement or off by 1 point.

303


When the two raters disagreed, they discussed theratings and evidence until they reached agreementon the rating. We show the indicators and eachschool's final ratings in Table 2.8 Because our def-inition of instructional program coherence devel-oped and refined as our analytic work proceeded,the rubric did not measure two final indicators-teacher hiring and evaluation practices-becausesuitable data were not available.

Statistical Analyses

The central question in our statistical analysesis whether schools with improving instructionalcoherence also show improvements in studentachievement. We employed a three-level latentvariable hierarchical linear model (HLM) to as-sess the relationship between changing levels ofinstructional program coherence and elemen-tary school achievement trends.9 We controlledfor both the initial school achievement level andthe initial level of instructional coherence. In thisway we could assess, regardless of where schoolsstarted, whether efforts to improve the coherenceof instruction culminated in improvements instudent learning. These analyses also took intoaccount other significant characteristics of theschools that were associated with both coherenceand student achievement trends.10 Thus we couldassess the effects of coherence net of these otherfactors, including size of school enrollment,racial/ethnic composition of a school, students'socioeconomic status, and the school's studentmobility rate."1

The Level I dependent variable was studentITBS scores, in logits (Rasch-equated ITBSscores). The unit of analysis was students, nestedwithin years, nested within schools. In the modelwe introduced adjustments for the various gradelevels taught in the elementary schools (at Level 1);possible time trend changes in student composi-tion (at Level 2); and other stable school char-acteristics (at Level 3). The latent variable analy-sis allowed us to control for differences amongschools in their base-year mean achievement aswe analyzed the effects of the changing levels ofprogram coherence on schools' test score trends.

The Level 1 (student-level) model was

Yijk = Hojk + rIljk (Grade 2) + rl2jk (Grade 3)

+ rI3jk (Grade 4) + l4jk (Grade 6)

+ fl5jk (Grade 7)

+ 1161k (Grade 8 dummy) + eijk

304

where Yijk is the ITBS score of student i in yearjin school k (j = 1993 ... 1997); ]?Iojk is the aver-age ITBS score of school k in yearj, controllingfor the percentage of students in each grade at theschool that year; H1jk..* X ll6jk are the average dif-ference in ITBS scores of each grade comparedto fifth grade, fixed across schools; and eijk is theunique error associated with student i (i.e., howstudent i is different from other students in hergrade in her school).

The Level 1 model produced a measure of meanachievement for each school for each year thatcould be fairly compared to those of other schools,regardless of the grade levels served. Each stu-dent's score was predicted with dummy variablesrepresenting grade levels (Grades 2 through 8,with grade 5 as the excluded group). The dummyvariables were grand-mean centered, and their ef-fects were fixed. As a result, the intercept fromthe Level 1 model, IlOjk, represented the adjustedmean student achievement for school k in yearj,controlling for the percentage of students at eachgrade level in that school in that year. The inter-cept from the Level 1 model served as the de-pendent variable in the Level 2 model:

rOjk = IOOk + Polk (Year)

+ 02k (% Low Income Students)

+ Rojk,

where IOOk iS the adjusted mean ITBS score forschool k in 1994; IOlk iS the average yearly growthin mean ITBS scores in school k from 1993 to1997; and 1 02k iS the relationship between the per-centage of low-income students and school meanITBS scores, fixed across schools.

The purpose of the Level 2 model was to pro-duce estimates of the yearly change in averageachievement for each school. The values for rojkfrom the Level 1 model were predicted with alinear variable representing the year and anotherrepresenting the percentage of low-income stu-dents in the school that year. The year variablediscemed the time trend in mean achievement forthat school. It was centered around 1994, where1993 =-1, 1994 = 0, 1995 = 1, and so on, so thatthe intercept, oOk of the equation, represented av-erage achievement for school k in 1994, and thecoefficient for the year variable represented theaverage yearly change in mean achievement inschool k from 1993 to 1997. The year prior to1994 was included as a base year so that anychange in achievement that had begun to occur


prior to measurement of coherence could be con-trolled. One time-varying demographic variable,percentage of low-income students, was enteredat Level 2 to capture large socioeconomic changesin the student body that might have occurredacross the years. Other demographic variableswere entered as controls at Level 3 because theywere measured on only one occasion, at the sametime as the initial measurement of school coher-ence, 1994.

The coefficients from the grade-level dummyvariables:at Level 1 (ljk .. . 16jk) were also pre-dicted with the year variable so that scores wouldnot be influenced by possible trends in studentperformance that differed by grade level (e.g.,rlljk = PIOk + PIlk * Year). Those effects were fixedacross schools.

At Level 3, the intercept, ook, and the slope as-sociated with the year variable, olk, from the Level2 models were predicted with variables represent-ing the level of coherence in the school in 1994,change in coherence from 1994 to 1997, andschool demographic and structural variables. Co-herence in 1994 was entered as a control variablein the prediction of changing achievement levels:

Pook = yooo + yo00 (coherence in 1994)

+ 7002 (mean student SES)

+ Y003 (African-American school)

+ 7004 (Hispanic school)

+ yoo5 (mixed minority school)

+ 7006 (integrated school)

+ 7007 (school size) + yO08 (mobility rate)

+ 7009 (change in coherence 1994-1997)

+ Uok

Polk 7010 + Yo0 (coherence in 1994)

+ 7012 (mean student SES)

+ 7o0j (African-American school)

+ 7014 (Hispanic school)

+ o015 (mixed minority school)

+ 7016 (integrated school)

+ 7017 (school size) + 7oi8 (mobility rate)

+ 7019 (change in coherence 1994-1997)

+ Yo110 (POOk) + Uok

To examine the relationship between changein instructional program coherence and change inmean ITBS scores, the coefficient of interest was

yoio. Because average growth in school achieve-ment may depend on student achievement levels'in the base year, the latent adjusted mean 1994achievement in school k, NUk,j was entered as a'predictor in the HLM model for the achievementtrend, lOk. Therefore, 7019 represented the rela-tionship between change in coherence and yearlygrowth in school mean ITBS scores, controllingfor the 'base achievement level of the school in1994. The control for 1994 achievement may haveproduced an overly conservative estimate of therelationship between change in coherence andchange in school improvement. However, it al-lowed for certainty that any observed relation-ship was not due to a relationship between schoolachievement in the base period and coherencechange (i.e., it might be more difficult for low-achieving schools to show improvement in schoolcoherence), or to improvement in test scores thatbegan before the first measurement of schoolcoherence.

Is Instructional Program CoherenceRelated to Student Achievement?

'We found a strong positive relationship be-tween improving coherence and improved stu-dent achievement. Schools 'that improved theirinstructional program coherence between 1994and 1997 demonstrated improved student testscores over the same period of time. There wasalso a positive cross-sectional relationship be-tween instructional program coherence and stu-dent achievement in 1994. The coefficients forthe cross-sectional relationship between coher-ence (on a 10-point scale) and student achieve-ment (in logits) were .078 in reading and .086 inmath. The coefficients for the relationship be-tween change in coherence from 1994 to 1997 andaverage yearly achievement growth were .012in reading and .014 in math.12 Results from theschool-level prediction of school achievement in1994 and the latent variable regression predictionof change in achievement from 1994 to 1997 arepresented in Table 1.13

Figure 1 graphically compares the change inmean ITBS scores among schools that declinedin coherence to those among schools that im-proved in coherence or stayed the same. Thenumbers in Figure 1 are based on the coefficientsin the bottom half of Table 1 but translated intoconcrete terms. For ease of interpretation, logitswere changed to "years of leaming," with 1 year

305

Newmann, Smith, Allenswort7z, and Bryk

TABLE 1HLM Coefficients: Prediction of 1994-1997 School Achievement Growth with Change in Coherence

Reading coefficients Math coefficientsDV Predictor Regular Standardized Regular StandardizedIntercept: Intercept -.423*** -. 150***1994 School Program coherence, 1994 .078* .095 .086* .108Achievement Student social status, 1994 .367*** .360 .302*** .308

African-American school, 1994 -. 140** -.135 -. 190*** -.190Hispanic school, 1994 .101 .052 .132 .071Mixed minority school, 1994 .298** .201 .264** .185Integrated school, 1994 .469*** .352 .469*** .366School enrollment, 1994 -.000 -.059 -.000 -.072Mobility rate, 1994 -. 003* -.119 -.002 -.082Change in coherence, 1994-1997 .077* .093 .064 .081

Slope: Yearly Intercept .023*** .023***Change in Mean Latent 1994 achievement -.001 -.027 -. 032** -.432Achievement Program coherence, 1994 .006 .199 .006 .1021994-1997 Student social status, 1994 .009 .242 .003 .041

African-American school, 1994 .002 .053 -.007 -.095Hispanic school, 1994 .003 .043 .001 .007Mixedminorityschool, 1994 -.017 -.313 -.009 -.085Integrated school, 1994 -.002 -.041 .011 .116Schoolenrollment, 1994 -.000 -.189 -.000 -.097Mobility rate, 1994 -.000 -.112 -.000 -.089Change in coherence, 1994-1997 .012** .398 .014** .238

*p <.05. ** p <.ol. ***p <.001

of learning calculated as .6 logits. Since we wereinterested in the change over a 3-year period,1994 to 1997, we multiplied the yearly effect by 3.As shown in Figure 1, on average, Chicago publicschools showed gains in test scores of 12-13% inboth reading and mathematics from 1993 to 1997.Schools that declined in coherence lost ground,however, relative to other CPS elementary schoolsover the same period. In contrast, schools thatshowed substantial improvement in coherenceachieved average ITBS scores that were almostone fifth of a year of learning higher in 1997than in 1994 (19% in reading and 17% in math).A test score gain of one-fifth year is equivalent toabout two additional months of schooling per year,which is substantial. It constitutes almost twiceas much improvement in test scores as that whichoccurred in schools with no change in instruc-tional program coherence, equivalent to about onemonth more schooling per year.

How Schools Achieved StrongerInstructional Program Coherence

Analysis of the field reports indicated substan-tial variation in instructional program coherence,

306

as is shown in Table 2. Three schools rated closeto the highest possible score of 52, but the totalratings for eight of the eleven schools were 33 orbelow, suggesting that strong instructional co-herence is not common among Chicago elemen-tary schools.

Schools that achieved stronger coherence ben-efited from principals and other key actors whodirected resources toward a clearly articulatedinstructional program framework. We now dis-cuss that leadership, the development of thoseframeworks, and some key strategies used tosupport stronger coherence among the field siteschools.

School Leadershipfor an Instructional Framework

Stronger instructional program coherence wasrooted in a principal's decision to adopt or de-velop a schoolwide instructional program frame-work and to make it a priority. We observed bothdemocratic and autocratic approaches to leader-ship that advanced such frameworks. All of theprincipals of the high-scoring schools shared aconviction that a more common approach to in-

Instrnctional Program Coherence

20 -

18 -

16 -

14 -

12 -

10 -

8-

6-

4-

2-

0-

19%

17%

13/--; 12/a

1 1 % I 0 r t > r--> 1 0 -O

5%

1994-1997 Average 1994-1997 AverageSchool Reading Improvement School Math Improvement

Change in coherence: 1994-1997

* Declined in coherence (10th percentile)

Li No improvement in coherence

'I Average improvement in coherence

.- Substantial improvement in coherence (90th percentile)

FIGURE 1. Growth in average school ITBS scores by change in coherence from 1994 to 1997 (controlling for

school demographics and coherence in 1994).

struction would assist student achievement. Notedone principal, "You will not have an effect if youare just working with five or six teachers; youhave to have 20 teachers to make a difference."

In two of the highest-scoring schools, Chelseaand Ackerman, principals led staff to collectivelyadapt and refine an instructional program frame-work. For four years Chelsea's principal workedwith an external partner and with teacher leadersto implement a framework for literacy diagnosis,development, and assessment based on the Read-ing Recovery program. Every teacher in the schoolreceived intensive training in the model. The prin-cipal made it clear that she wanted all teachers,regardless of grade, subject, or bilingual assign-ment, to work with a specific conception of liter-acy development and to collaborate with peers inusing it. The collaboration was more explicit thanan agreement among teachers to "teach narrativeand expository writing." Instead, it required teach-ers to agree on which materials to use, which in-

structional strategies to apply to distinct forms ofwriting, how to formulate grade-level expecta-tions, and how to recognize various stages of ac-complishment. In this way and others, Chelseateachers were supported and prodded to work be-yond their own classrooms in order to implementa systemic conception of instruction and literacydevelopment.

Leadership at Ackerman also emphasized thedemocratic establishment of an instructionalframework. Ackerman served students from ex-tremely poor homes. Due to chronically low stu-dent achievement, the school had been on acad-emic probation for several years. Consequently,the staff decided to adopt a direct instruction (DI)program modeled on the DI Program used inHouston, Tex., which some staff had traveled toobserve (Adams & Engelmann, 1996; AmericanFederation of Teachers, 1998). Explained one staffmember, "We had a problem with our test scores.

(Text continues on page 310)

307

J 0)

Cl).o -

Co

0) a

o xoz 0uz "

CD co

E) (D

= N

N N N =

CQ N

Ns

N

N =

CN4 N CN

C CN cN

N Cl) N

n t N

0 0 C

Ca 0 0 CC Ca . .~~~~~~~~~0

0 0C C) 0 C.) ~ ~ ~ ~ ~ ~ ~ 0 o d. )

C) C)i to C >.10a 1) --Ca Ca > >

0 o, 00

CZ wC 0d U.) E0 z~~C CC 0 Oa~ C C)C .

Ca -s CZ CaC-~~CaOCOC)CaC)Ca >.~ . 0 4 C)C).~ 0 C)Q Ca 0 Ca

0.

=4 c'i c.i 4I

r0

c0

Sizm

C)

ir.

0

Cd

C)

.0

..0

0

C.)1

0

m

C.)X0

0

C4

CZ,0Z0z

.P

C.

C)

C.

CO

i0

2C.

Q~

V)i vo r-

cl Cl1 Cl cl

Cl \OCl

Cl l-

cl Cl C t ,t 00Nf C

Cl4 0

Cl4 l clq

Cl Cl

i .Y t .=m O " i o U

oo o0 o o 0 U

~0i

00 0\~~ = C

ci

00

U8

09

~00

0i

PV

0

zm,

cli

iiI

4


We came together as a group to decide on thebest method to get us out of the hole we were in.Direct instruction has worked and we are stillworking with it." To supplement the direct in-struction program, Ackerman funded training anddevelopment of eight grade-level teacher-literacyleaders, who helped peers to build classroom li-braries from which students would regularly selectbooks for reading and writing. The fully articulatedinstruction and assessment scheme provided by thedirect instruction program, along with schoolwideimplementation of the literacy/library supplement,were leading reasons for Ackerman's high ratingon the field study rubric. Every teacher in theschool was trained in these frameworks and usedthem daily. Not everyone would agree with Ack-erman's decisions to adopt a direct instructionframework. But staff considered the shared in-structional framework of specified strategies forteaching and assessment a significant improve-ment over past practices.

Not every school developed instructionalframeworks democratically. In Bishop, and to alesser extent Hartford and Larkin, principals oradministrative teams mandated that teachers usepurchased frameworks; teacher responses weremixed. Nonetheless, those schools had highly-to-moderately coordinated instructional programsemphasizing shared instructional strategies andassessments, as well as sustained staff develop-ment aimed at consistent implementation of theframeworks.

Principals of the less-coherent schools didnot seek to organize their instructional programsaround common frameworks, and their schoolswere characterized by norms of individual teacherautonomy over curriculum materials, instruc-tional strategies, and assessment. For instance, theprincipal at Sparrow School, intending to supportteacher motivation and innovation, was reluctantto constrain teachers' approaches to instruction.She explained, "I don't say no to many people orto many things.... We try to bring in new ideasand new methods, new strategies." Approachesto literacy at Sparrow differed from classroomto classroom and included whole-language ap-proaches, direct instruction approaches, and arange of mixed practices.

Another school with low coherence scores wasWilson. Located in an economically mixed Chi-cago neighborhood, the school was well regardedby parents and the community. Wilson staff

310

clearly shared a priority goal-to improve theirstudents' standardized test scores. They had manyprograms and supports that could conceivablyhelp them reach that goal, but the resources werenot organized into an instructionally coherent pro-gram for students or teachers. Classroom obser-vations and follow-up interviews rendered littleevidence of a framework beyond a curriculum ofitems similar to those on the standardized tests. Itwas up to each teacher to decide what parts of sucha "curriculum" to teach and how to teach them.Different curriculum materials and instructionalapproaches were used both within grades andacross the school. The statement by one teacherthat "[t]eachers here have the same frustrations, butthey deal with them independently in their classeswithin the context of the [test] goal" was reiteratedin similar terms by most members of the staff.

In other low-coherence schools, weak instruc-tional program coherence reflected not simplythe lack of an instructional program frameworkbut diffuse and uncoordinated instructional lead-ership (Mathews, Templeton) or a clear lack ofleadership (VanDyke). Many staff members inthose schools seemed wary of using any singleinstructional framework, feeling that because"some kids thrive on one method and some onanother method" it was best to allow teachers asmuch choice and discretion as possible

In one school the principal's imposition of acommon instructional framework had negativeconsequences. Bishop elementary school, whichscored high on both survey and field-based mea-sures of instructional program coherence, hadmuch in common with Ackerman. It enrolled stu-dents from a very impoverished community, hadbeen on academic probation since the sanctionwas first implemented, and had adopted a directinstruction program to focus instruction through-out the school. Many at the school saw studentachievement gains from 1994 to 1997 as a clearoutcome of this approach. But Bishop was a verytroubled organization. A large number of teachersopposed the exclusive use of the direct instructionmethod. As one teacher observed, "Direct instruc-tion wasn't something the teachers selected. It wasselected for us." Researchers reported teachers'perceptions that the principal had become in-creasingly autocratic and distant, with a wan-ing commitment to school improvement. Severalteachers reported declining motivation to worktogether because they saw little opportunity to


voice or implement their ideas. Other teacherswere demoralized and looking for work in otherschools. Bishop raises an important point. Strongschool-level leadership is central to the develop-ment of stronger instructional program coherence,but coherence achieved by administrative fiat isof questionable value when it suppresses the de-velopment of equally essential supports for learn-ing, such as teachers' professional community andshared ownership of an instructional program. Weprovide additional cautionary notes on the devel-opment of coherence later in this article.

Partner Support for Coherence

A school principal's actions to institutionalizea common instructional framework are central tostronger coherence, but other actors can also pro-vide leadership that promotes coherence. As par-ticipants in the Chicago Annenberg Challenge,each of the fieldwork schools was linked to anextemal partner responsible for helping the schoolto improve. Chelsea, which adopted its partner'sprogram as the central strategy for school improve-ment, showed that an extemal partner can signifi-cantly assist a school to achieve stronger instruc-tional program coherence. The school's partnerwas instrumental in training the entire staff in theliteracy framework, training full-time literacy co-ordinators, helping to develop a schoolwide lead-ership team that assumed ownership of the schoolimprovement model, and helping teachers to care-fully analyze their own progress and their stu-dents' progress. This focused, long-term partner-ship was not typical, however, and enhancinginstructional coherence was not a central objec-tive for most partners in the fieldwork sample.More commonly, partners guided the develop-ment of particular programs or initiatives, for ex-ample, parent involvement or the integration of artsinto the curriculum. In many of the low-scoringschools, some teachers had no idea their school hadan external partner. Several schools had multi-ple external partners who never coordinated theiractivities (Newmann & Sconzert, 2000).

Organizing Resources to Strengthen Coherence

The more coherent field study schools were notsmaller, did not have significantly greater fiscalresources, and did not have significantly morestaff members per student than those with lower-rated instructional program coherence. Rather, thehigher-rated schools used a core set of strategies

to harness available resources and staff energytoward a common instructional framework andpriority school improvement goals. Those strate-gies included (a) investing in technical resourcesthat assisted whole-school development, (b) fo-cusing staff collaboration on the common instruc-tional framework, and (c) channeling communityresources to support a core instructional program.

Investing in schoolwide useof technical resources

A feature that distinguished the high- and low-coherence schools was significant investment ininstructional materials and programs, includingstaff training and development, that were groundedin a particular conception of instruction and-perhaps more important-implemented school-wide. Ackerman andlBishop invested in direct in-struction programs that provided published,comprehensive frameworks across subjects andgrades as well as all necessary materials forlessons and staff training. In contrast, Chelsea'scomprehensive literacy framework was morecomplex. It offered more flexibility in daily in-struction and instructional materials, called for agreater range of technical resources, and de-manded more extensive training. Hartford's ap-proach was somewhat between the two models,involving teachers in continuing workshopsand training to deliver a program of commoncurriculum units and assessments but lacking spe-cific training in instructional strategies.

To foster progress and staff commitment, morecoherent schools also funded program coordina-tors who directly assisted staff to implement in-structional frameworks. Again, the strongest ex-ample of this was at Chelsea, 'whose facilitatorsoffered after-school workshops, searched for ma-terials, outfitted the resource center that supportedthe program, and routinely visited classrooms toobserve, demonstrate, coach, and co-teach. Butseveral other schools (Ackerman, Bishop, Larkin,and Mathews) also funded facilitators. Sincehighly trained coordinators are often recruitedaway from the schools that train them, schools'investment in such positions, while pivotal, canalso be risky. We noted, for example, that low-scoring VanDyke had invested heavily to trainand develop literacy coordinators, who left theschool soon after completing the training.

The mid- and low-scoring schools did not usetechnically strong whole-school models and did

311


not employ well-trained school coordinators tooffer continuing staff development. Instead, theytended to use more diffuse assistance and de-velopment strategies. For example, Larkin hadmade gains in coherence by instituting more com-mon and coordinated curriculum and assessmentmaterials. But it also purchased multiple read-ing programs and workshops that each involvedsmal numbers of teachers. In other low-scoringschools (Sparrow, Templeton, and Wilson) prin-cipals spoke of making assistance "available" toteachers and inviting teachers to "ask for what theyneed." Many teachers in these schools partici-pated extensively in school improvement activ-ities, but their efforts were not focused and co-ordinated. In several cases, teachers reported thatthey were unaware of the improvement activitiesof fellow staff. Low-scoring schools with morediverse programs and investments also appearedto have greater problems following through onresources in which they had invested. Several hadpurchased materials and computer technology thatthey believed could help students perform betteron standardized tests. But these purchases weredelivered without training, discussion, or assess-ment of how to maximize or evaluate their bene-fits and without a scheme for ensuring that theywould be used in the manner intended. Only someteachers reported actually using the materials.

Staff collaboration within a common framework

Tighter coordination of instruction requiresmore than commitment to a schoolwide frame-work grounded in a strong technical base. Itrequires extensive, continuing communicationamong teachers, mutual assistance, and workingtogether to improve instruction according to theframework. School leaders in the higher-scoringschools promoted such collaboration by estab-lishing common planning periods for grade lev-els and across grade-planning groups, by forming"schools within schools" to facilitate more exten-sive collaborative relationships through smallerclusters of teachers, and by establishing represen-tative teams and committees to deal with schoolgovernance issues. It is important to note that thestructures alone were not sufficient to cultivategreater coordination and coherence. Rather, prin-cipals, program coordinators, and others workedto coax teachers into collaborative activity aroundcore instructional goals and strategies. In schoolswith lower coherence scores, planning periods

312

were not available or were seldom used for groupwork within a common instructional framework.Staff at one low-scoring school had recently aban-doned common planning periods because few ofthe teachers used the periods for collaborativework and the periods complicated the schedule.

Coordinating community resources

Principals also faced the challenge of how to se-lect and coordinate social resources, such as parentvolunteers and programs sponsored by communityagencies. Such programs seemed responsive tostudent needs or provided additional funds, orboth, but staff comments often suggested a needto "get a better handle" on the programs. Mostschools worked to mobilize parent involvementprograms to emphasize reading and homeworkassistance. Because assistance of that kind wasclearly connected to undisputed literacy goals,it generally posed no threat to instructional pro-gram coherence, though some programs weremore strongly linked than others. But the im-pact of many community-sponsored programs oncoherence was less clear, for example, adult-student mentoring, business partnerships, envi-ronmental projects, and museum and arts outings.Such programs, typically designed as short-term,stand-alone initiatives for adoption by any school,seemed less likely to connect to a school's partic-ular instructional framework. Several principals inthe mid- and low-scoring schools said that theyrarely rejected such programs because they sawthem as valuable opportunities to enlarge stu-dents' horizons and boost self-esteem throughnonacademic pursuits. Whether such programsweaken instructional program coherence dependson how they are administered and managed. But if,as was observed in several of the low-scoringschools, they disrupt teachers' and students' pro-gress on the main instructional framework, theyweaken coherence and its potential outcomes.Principals in the higher-scoring schools seemedmore ready to minimize such threats and to ac-knowledge that "[y]ou can, in fact have too manyresources."

Summary Review of Empirical Findings

Our research results suggest that school im-provement efforts that strengthen instructionalprogram coherence can lead to increased studentachievement. Our quantitative analysis usingteacher survey responses shows positive connec-


tions between strengthening instructional pro-gram coherence and improving academic achieve-ment in both reading and mathematics in a largenumber of schools. And correspondence betweenteacher survey responses and ratings by inde-pendent observers indicates that the strength ofprogram coherence in individual schools can bereliably measured.

Schools in our field sample that ranked higheron instructional program coherence had strongerprincipal leadership than did lower-rankingschools. With the help of other'instructional lead-ers and extemal partners, these principals insti-tuted common instructional frameworks andsupported them by investing in schoolwide tech-nical resources such as high-quality curriculumand assessment materials. They promoted ex-tensive collaboration among staff and focusedcommunity-based programs and resources on oneor a few core schoolwide improvement goals.They sustained these efforts over a significantperiod of time (three or more years and count-ing). In contrast, leaders in lower-ranking schoolsoften believed that schoolwide commitment toimproved test scores was a sufficient mechanismfor improved practice. New resources and col-laborative effort in those schools were often sig-nificant but were spread across multiple pro-grams and initiatives, each involving a limitedgroup of faculty.

Cautions in the Pursuit ofStrong Instructional Program Coherence

It is important to place strong instructional pro-gram coherence in perspective and to understandit not as the overriding task of school improvementor as a substitute for key supports for teaching andlearning, but as a strategy for maximizing thosesupports. Strong instructional program coherencecould, for instance, undermine the developmentof teachers' professional community if it insistedon such regimented instruction that teachers hadno opportunity to exercise expertise or raise ques-tions about selected methods or programs-ashappened at Bishop. Similarly, teachers' profes-sional development opportunities might be madeso uniform as to prevent individuals from learn-ing skills unique to their teaching situation orbackground. The pursuit of greater program co-herence must respond to appropriate forms of dif-ferentiation and be receptive to new or altered pro-granmming for staff and for students when clearly

necessary. Program. stability is desirable becauseit allows for continuing learning and improve-ment. But the benefits of stability should not beinvoked to sustain ineffective programs.

Moreover, our theoretical explanations of howinstructional program coherence promotes stu-dent achievement could apply to many types ofcurriculums and instructional programs. A schoolcould become'highly coherent (and could evenincrease student scores on'standardized tests) byinstituting instructional frameworks that are nar-rowly focused on the most rudimentary academictasks. But it would still be perceived by some ormany as a school that denies students necessaryopportunities for individual expression, higher-order thinking, or comprehensive understand-ing of a variety of subject areas-as at Bishop.Thus the ultimate value of strong program co-herence will always depend on the perceived ed-ucational legitimacy of what students leam andhow they learn it.

Finally, because we did not attempt to test acomprehensive model of all the factors that con-tribute to student achievement, but examined onlythe influence of instructional program coherence,we cannot say whether it is more or less importantthan other school-level variables that may influ-ence student achievement, such as curriculumcontent, teacher expectations for students, or thestrength of professional community among teach-ers. The relative influence of these variables ascompared with instructional program coherence issimply not known. Based on our research we can,however, recommend that educators work to es-tablish instructional program coherence as theyseek to strengthen other supports.

Why It Is Difficult to Achieve StrongInstructional Program Coherence

Even when instructional program coherence ispursued in the context of other supports for schoolimprovement, our data suggest that many schoolswill have difficulty. To understand'the challengesinvolved in strengthening instructional programcoherence, we outline several reasons that manyschools have relatively weak coherence, based onour field studies, other studies of school improve-ment (Bryk, Sebring, et al., 1998; Newmann &Wehlage, 1995), the policy coherence literature(Cohen, 1995; Smith & O'Day, 1991), and aliterature synthesis on school capacity'building(King & Newmann, 1999). These sources include

313


explanations of weak program coherence thatpoint to factors operating both within the schooland beyond.

Factors Within Schools

Lack of consensus within a school in efforts toestablish a clear focus often reflects demands formultiple and varied learning outcomes, such asgood behavior, basic skills, advanced conceptualunderstanding, and higher self-esteem. Moreover,student diversity is usually assumed to requiredistinct program interventions for each group orproblem (e.g., for students who are gifted-talented,at-risk, speak English as a second language, orneed special education). This differentiation isreinforced by staff specialization and by categor-ical funding aimed at special groups or problems.The press toward separate programs for differentlearning goals and different students makes itdifficult for teachers to work from a commoninstructional framework.

In addition, winning staff acceptance of a com-mon instructional framework is often seen as anoverwhelming task, because it requires continuingagreement, cooperation, and training on the part ofboth old and new staff. Incremental improvementinvolving small segments of the staff who worktoward short-term goals seems more adminis-tratively manageable than trying to achieve long-term, schoolwide instructional coherence.

Finally, an uncertain knowledge base abouthow to effectively teach and assess student leam-ing tends to encourage a trial-and-error approachrather than a common, coordinated approach to in-struction. Unlike fields where research has pro-duced highly reliable methods of diagnosis andintervention, such as medicine or engineering, ed-ucators face substantial uncertainty about how toproceed. Under such circumstances, it can seemreasonable to give each teacher considerable lati-tude in deciding how to teach rather than requiringa common approach. Although this perspective hasmerit, our evidence indicates that achieving coher-ence around a defensible framework does matter.There may not be one best framework for organiz-ing instruction in all schools, but the presence of aschool-specific framework can enhance studentand teacher learning.

Factors Beyond Schools

Ideally, strong school leadership would steer aschool toward greater instructional coherenceand minimize the influence of the other factors

314

mentioned earlier. But tendencies toward inco-herence are reinforced by an increasing numberof independent providers of school assistance,such as reform project leaders, professional de-velopment facilitators, technology consultants,and instructional material vendors. Incoherenceis also aggravated by unaligned district and statepolicies and by rapid changes among them.

Independent providers of improvement programsand materials

Independent providers typically establish theirimpact and legitimacy by disseminating discrete,identifiable programs to many schools, not byhelping individual schools to develop focused, co-herent missions. Foundations, universities, andother providers usually have an interest in a partic-ular program or topic, such as early childhood de-velopment, literacy, or classroom management. Inour visits to schools, for example, we observed in-stances in which facilitators from various organi-zations worked extensively with a school staff butnever talked or worked with one another to help theschool focus and integrate their contributions.

Another obstacle is the nature of the educationpublishing industry. Education publishers tend tobe rewarded for producing materials for sepa-rate subjects and distinct instructional approaches,rather than for comprehensive series integratinghigh quality materials for multiple subjects andgrades. Some publishers offer coordinated ma-terials for individual subjects, most commonlyK-6 language arts programs. However, integrationof instructional approaches across subjects israre. Schools must shop strategically across mul-tiple vendors to patch together an instructional pro-gram that includes some common elements.

Finally, both independent providers and pub-lishers must recruit schools to join their effortsor to purchase their products and services. Theyoften have minimal political or economic powerto influence schools; as vendors in a market theymust essentially persuade schools to let them in.To gain access and the trust needed to work witha school, an assistance organization or publishermust, in large measure, accept the school's prior-ities rather than insist that the school work towardstronger instructional program coherence.

School districts and states

Compared to independent providers of im-provement programs and professional develop-


ment, districts and states conceivably have moreclout to strengthen school instructional programcoherence. But for several reasons, governmen-tal authority is not exercised to achieve strongercoherence. Because debate persists on the degreeof authority that parents, teachers, schools, schooldistricts, and states ought to have to prescribeeducation programs, the actual power of districtsand states to control school affairs is contested(Allington & Johnson, 1989; Bryk, Sebring, et al.,1998; Cohen, 1995; Fuhrman & Malen, 1991).For example, when schools choose to work witha variety of external partners, neither district norstate policy requires that the partners coordinatetheir efforts within schools. And wheh districtsand states themselves offer professional devel-opment, participation by schools is often volun-tary, in which case the district or state occupies aposition similar to that of independent providers.Districts routinely offer a variety of professionaldevelopment options from which individual teach-ers can choose, but rarely with a requirement orincentive for entire school staffs to pursue train-ing together. Even when districts mandate pro-fessional development, programs are seldomdesigned to help school faculties develop under-standing of and commitment to a common in-structional framework.

One area in which districts and states have re-cently asserted authority is the promulgation ofstandards for curriculum and assessment. Yet thestandards often consist of voluminous lists ofdiscrete skills and items of knowledge and fail tocommunicate a common framework for instruc-tion. Professional development aimed towardmeeting dozens of standards for various subjectsat various grade levels offers no common mis-sion for a school except that each teacher teachwell the list of specified items in each subject andgrade level. Grade-level teams may work togetherto implement the standards, but such efforts aredistinct from professional work focused on acommon instructional framework that emphasizesconnections between subjects or grade levels.

These observations are consistent with histor-ical and sociological analyses of how the exter-nal environment influences schools (Rowan &Miskel, 1999; Tyack & Cuban, 1995). Researchhas shown, for example, that since the 1960s, fed-eral and state governments have asserted greaterauthority and increased centralization. But at thesame time, local communities have increased their

authority through school site management. Thelack of policy coordination between these levelsexacerbates fragmentation. Thus weak programcoherence is duepartly to a system of political con-trol that resists tight coordination and that in-cludes frequent changes of leadership at all levels(Cohen, 1995; Hess, 1999). As a result, schools, inresponding to their external policy contexts, havebecome far more complex organizations withincreased funds. and personnel dedicated to man-agement and compliance (Scott & Meyer, 1994).Given the extent to which administrative resourcesmust concentrate on managing complex organiza-tions and a multitude of external demands andcontexts, it is not surprising: that instructional pro-gram coherence suffers.

Implications for Education Leadersand Policymakers

The most straightforward implication of thesefindings is that leaders in schools, in school im-provement organizations, and in district, state, andfederal agencies should give more deliberate at-tention to strengthening instructional program co-herence within schools. The following activitiescould be helpful in various organizational settings:

* School principals could focus their improve-ment plans, professional development, and ac-quisition of instructional materials on a few coreeducational goals pursued through a common in-structional framework. Shared schedules and sta-ble teaching assignments could provide commonplanning time and sustained opportunity to buildskills in implementing the framework. Teacherhiring and evaluation could emphasize skillfuluse of the framework. Grants and partnershipscould be limited to those that contribute to im-plementation of the framework.

* Foundations and other organizations that sup-port school improvement could emphasize co-ordination of improvement efforts within schools.A foundation that supplies funds for school im-provement could require separate projects in lit-eracy, math, and science to coordinate their ef-forts within each school. Improvement programscould be directed to clarify how recommendedpractices reflect common approaches to instruc-tion and how teachers will have sufficient time tolearn, adopt, and refine them. Staff developmentproviders could insist on woriting with schoolteams rather than a few individual teachers and

315


could structure the work toward implementationof effective practices across grade levels ratherthan only in selected classes.

* District policy could emphasize instructionalprogram coherence as a key dimension of schoolimprovement plans. District-sponsored profes-sional development could be organized aroundschool teams that work on common instructionalframeworks. Criteria for hiring and evaluation ofprincipals and professional development for lead-ers could emphasize instructional program coher-

.ence. Administration of categorical aid programsand approval of extemal partnerships with schoolscould require coordination among programs. Dis-trict curriculum standards and tests could be re-vised to reflect greater integration of learning out-comes across subjects and grade levels. Finally, anoversight committee could review district man-dates and regulations to consider their effects oninstructional program coherence within schools.

* Administration of state categorical aid pro-grams could require that funds be used to supportnot just schoolwide programs but schoolwide pro-grams that are focused and sustained over multi-ple years and that serve a clear instructional frame-work. State curriculum standards and tests couldbe revised to encourage greater integration oflearning outcomes. An oversight committee couldreview state mandates and regulations to considerwhether schools could become more coherentunder such mandates.

Conclusion

In searching for ways to improve failing ele-mentary schools, many reformers and school lead-ers resolved that adopting a range of new ideas andprograms was the best strategy to spur positivechange and to discover what works. The hope wasthat program variety, teacher choice, and incre-mentalism would combine to produce meaningfulimprovements. This article addresses the reasonsthat such a strategy may not often produce cumu-lative benefits. To improve student achievement,school staffs and the extemal organizations thatwork with them should aim to strengthen instruc-tional program coherence. They can do this byworking together to recognize both intemal andextemal sources of incoherence and by directingresources to the development of a common in-structional framework that guides and coordinatessupports for teaching and learning throughout theschool.

316

Since the sources of incoherence rest bothwithin and beyond schools, strengthening instruc-tional program coherence requires simultaneouseffort from the bottom up and from the top down.Principals and teachers can be supported by effortsto minimize barriers to coherence that emanatefrom district, state, and federal school govemancebodies and a growing market of independent ser-vice providers. If actions to strengthen instruc-tional program coherence are integrated withefforts to develop other key supports for schoolimprovement, schools can build and reinforceforms of staff competence and commitment thatadvance student learning.

APPENDIX

Program Coherence Rubric

1 Teachers within a grade purposely link theircurriculum (including arts, health, library,and computers) to learning goals.1 = Teachers within a grade do not connect their

curriculum to these goals.2= Some teachers within a grade connect some

of their curriculum to these goals.3 = All teachers within a grade connect some of

their curriculum to these goals.4 = Teachers within a grade connect their entire

curriculum to these goals.2 Teachers within a grade use common instruc-

tional strategies.1 = Teachers within a grade do not use common

instructional strategies.2 = Some teachers within a grade use some

common instructional strategies.3 = All teachers within a grade use some com-

mon instructional strategies.4 = All teachers within a grade use many com-

mon instructional strategies.3 Teachers within a grade use common assess-

ment strategies.1 = Teachers within a grade do not use common

assessment strategies.2 = Some teachers within a grade use some com-

mon assessment strategies.3 = All teachers within a grade use some com-

mon assessment strategies.4 = All teachers within a grade use many com-

mon assessment strategies.4 Teachers coordinate curriculum and assess-

ments to avoid repetition and to offer studentsnew and more complex aspects of subjectmatter as they move from grade to grade.1 = Teachers do not coordinate curriculum and

assessments across the grades.


2 = Some teachers coordinate some curriculumand assessments across the grades.

3 = All teachers coordinate some curriculumand assessments across the grades.

4 = All teachers coordinate all curriculum andmany assessments across the grades.

5 School-sponsored support programs, such asremedial instruction, assemblies, field trips,tutoring, and parent education, are linked tothe curriculum, instruction, and assessmentsof the school program.1 = Few to no support programs are linked to the

curriculum, instruction, and assessments ofthe school program.

2 = Some support programs are linked to thecurriculum, instruction, and assessments ofthe school program.

3 = Most support programs are linked to the cur-riculum, instruction, and assessments of theschool program.

4 = Almost all support programs are linked to thecurriculum, instruction, and assessments ofthe school program.

6 Professional development for staff supportsthe implementation of common curriculum,instructional strategies, and assessments.I = Professional development for staff does

not support the implementation of commoncurriculum, instructional strategies, and as-sessments.

2= Professional development for staff onlyweakly or sporadically supports the imple-mentation of common curriculum, instruc-tional strategies, and assessments.

3 = Professional development for staff regu-larly supports the implementation of com-mon curriculum, instructional strategies, andassessments.

4 = Professional development for staff exten-sively supports the implementation of com-mon curriculum, instructional strategies, andassessments.

7 Professional development programs are sus-tained over time.1 = No professional development programs (or

almost none) are sustained over time.2 = Some professional development programs are

sustained over time.3 = Many professional development program are

sustained over time.4 = Almost all professional development pro-

grams are sustained over time.8 The school strategically accepts and refuses

programs and initiatives in a manner that sup-ports staff focus, program continuity, and on-going improvement.

1 The school has many programs, and staff en-ergy and attention are very fragmented. Thusthe programs do not support staff focus, pro-gram continuity, or ongoing improvement.

2= The school has many programs, but some at-tempts are made to focus staff energies andattention to support program continuity andongoing improvement.

3 The school is somewhat strategic in how itaccepts and refuses programs so that theymay support staff focus, program continuity,and ongoing improvement.

4 The school is very strategic in how it acceptsand refuses programs so that they may sup-port staff focus, program continuity, and on-going improvement.

9 School improvement planning and assessmentdirectly address the school's progress in pro-viding a conmnon, coordinated, and sustainedschool program.1 = School improvement planning and assessment

generate long, disconnected lists of goals andassessments.

2 = Some school improvement planning and as-sessment addresses the school's progress inproviding a common, coordinated, and sus-tained school program.

3 = Much school improvement planning and as-sessment addresses the school's progress inproviding a common, coordinated, and sus-tained school program.

4 = A great deal of school improvement planningand assessment addresses the school's pro-gress in providing a common, coordinated,and sustained school program.

10 Curriculum remains reasonably stable overtime and thus provides teachers with sustainedopportunities to learn how to teach it well.1 = Curriculum is very unstable and frequently

changes. Teachers have no sustained oppor-tunity to learn how to teach it well.

2 = Curriculum is onry partially stable over time.Teachers have sustained opportunities to learnhow to teach some of it well.

3 = Curriculum is fairly stable over time. Teach-ers have sustained opportunities to learn howto teach much of it well.

4 = Curriculum is very stable over time. Teach-ers have sustained opportunities to leam howto teach most of it well.

11 Assessment remains reasonably stable overtime and thus provides teachers with sustainedopportunities to teach students how to succeedon them.1 = Assessment is very unstable and frequently

changes. Teachers have no sustained oppor-

317

Newmann, Smith, Allenswortlz, and Bryk

tunity to teach students how to succeed onthem.

2= Assessment is only partially stable overtime. Teachers have sustained opportunitiesto teach students how to succeed on some ofthem.

3 = Assessment is fairly stable over time. Teach-ers have sustained opportunities to teach stu-dents how to succeed on many of them.

4 = Assessment is very stable over time. Teach-ers have sustained opportunities to teach stu-dents how to succeed on most of them.

12 Teaching assignments remain stable over time.1 = Teaching assignments are very unstable and

frequently change.2 = Some teaching assignments are stable while

others frequently change.3 = Many teaching assignments are stable over

time.4= Most teaching assignments remain stable

over time.13 Key program leaders or leadership positions

remain stable over time.1 = Key program leaders or leadership positions

are very unstable over time and frequentlychange.

2 = Some key program leaders or leadershippositions are stable while others frequentlychange.

3 = Many key program leaders or leadershippositions are stable over time.

4 = Most key program leaders or leadershippositions are stable over time and rarelychange.

Notes

The authors are grateful for the assistance of manyorganizations and individuals who contributed to thiswork, including Mark Smylie, Stacy Wenzel, JohnEaston, and other members of the Lead Team of theChicago Annenberg Research Project, Cora Marrett,Deborah Stipek, and Leona Shauble, and staff from theChicago Public Schools. We would also like to thankthe anonymous reviewers for their thoughtful advice.This work was supported by the Chicago AnnenbergChallenge.

' This study is limited to elementary schools. Weacknowledge that our approach to coherence may bemore important to student achievement in elementaryschools than in high schools, where within-subjectcoherence may be more important than a commonframework applied across subjects. We hope that fu-ture research will investigate the importance of in-structional coherence at various levels of schooling.

2 The collected tasks and work samples were notpart of our coherence analyses and are not elaborated

318

on here. For more infornation see Newmann, Lopez,and Bryk, 1998.

3 In 1997, the mean squared fit statistics for thecommon items were .61, .87, 1.1, and .93; those for thenew items were .81, .99, 1.05, .76, and 1.03.

4 We also constructed a 1997 measure of coher-ence without the additional items and compared thatmeasure to those used in the analysis. It had a highcorrelation of .88 with the full 1997 measure of co-herence, and its correlation with coherence in 1994(.52) was almost identical to the correlation of thecomplete measure in 1997 with the 1994 measure (.53).We decided to use the full 1997 measure because ofthe increased reliability obtained by including theadditional items.

5 The 1997 measure was more reliable because ofthe additional items.

6 Empirical Bayes estimates were produced throughthree-level hierarchical linear models, in which thefirst level was a measurement model, the second mea-sured teachers, and the third measured schools.

7 Official documents on curriculum and school im-provement planning were valuable sources of school-level data, but the indicator ratings reflected whatresearchers observed and what they leamed during in-terviews. For example, to assign a high score on the in-dicator "teachers within a grade use common assess-ment strategies," the researchers had to see or hearevidence of use of common assessment strategies fromall of the observed and interviewed teachers.

8 School names in Table 2 and in the text arepseudonyms.

9 For details on latent variable HLM, see Rauden-bush, Bryk, Cheong, and Congdon (2000), p. 207.

10 In both 1994 and 1997, there were significantnegative relationships between instructional programcoherence and school size, the percentage of low-income students, and the percentage of non-White stu-dents. Class size showed a positive relationship withcoherence, but it was not a significant predictor ofstudent achievement and was not included in the finalmodel. Instructional program coherence was not sig-nificantly related to the experience of teachers in theschool in either year, so teacher experience was not in-cluded as a variable in the model. Coherence had anonsignificant positive relationship with teachers' ed-ucation (the percentage of teachers with a Master's de-gree) in 1994 (r = .08), and a marginally significantnegative relationship in 1997 (r =-.10), but it was notrelated to student achievement and so was not includedin the final model. Additional models were run withthese excluded predictors (class size, teachers' educa-tion) to be sure they would not change our inter-pretation of the results. Their inclusion had virtuallyno effect on the coefficients representing coherence-achievement relationships.

Instructional Program Col

11 We do not include control variables for organiza-tional characteristics or for types of instruction be-cause we are examining the effect of instructionalprogram coherence regardless of the content of in-struction or of how coherence is achieved. We expectthat efforts to increase program coherence will affectinstruction, teacher leaming, and other organizationalcharacteristics, and we do not want to disregard thoseeffects. This analysis provides the first empirical evi-dence that there is a connection between instructionalprogram coherence and;student achievement. Furtherstudies may examine in more detail the mechanismsthrough which this relationship is achieved.

12 An analysis of residuals was performed to be cer-tain the relationships could not be attributed to out-liers. This was confirmed. Only 5% of the cases ineach analysis had standardized residuals less than -2or greater than 2, and only one case in each analysishad a standardized residual greater than 3. No case hadsubstantial leverage.

13 Standardized coefficients were produced forTable 1 so that effect sizes could be discerned amongpredictors. Standard deviations for the two depen-dent variables (1994 achievement and yearly changein achievement from 1994 to 1997) were calculatedthrough HLM models that were unconditional atLevel 3 and had only the "year" predictor at Level 2.Grade-level dummies remained in these models be-cause they were necessary for production of measuresof mean school achievement that were independent ofgrade levels served by the school. Standard deviationsfor the independent variables were based on school-level statistics.

References

Adams, G. L., & Engelmann, S. (1996). Research ondirect instruction: Twenty-five years beyond DIS-TAR. Seattle: Educational Achievement Systems.

AUington, R. L., & Johnson, P. (1989). Coordination,collaboration, and consistency: The redesign of com-pensatory and special education interventions. InR. E. Slavin, N. L. Karweit, & N. A. Madden (Eds.),Effective programsforstudents at risk, pp. 320-354).Needham Heights, MA: Allyn & Bacon.

American Federation of Teachers. (1998). Building onthe best, learning from what works: Seven promis-ing reading and English language arts programs.Washington, DC: Author.

Ames, R., & Ames, C. (Eds.). (1984). Research onmotivation in education. Orlando: Academic Press.

Blauner, R. (1964). Alienation andfreedom: The fac-tory worker and his industry. Chicago: Universityof Chicago Press.

Bransford, J. D., Brown, A. L., Cocking, R. R. (Eds.).(1999). How people learn: Brain, mind, experience,and schooL Washington, DC: National AcademyPress.

Bryk, A. S., Easton, J. Q., Kerbow, D., Rollow, S. G.,& Sebring, P. A. (1993). A view from the elementaryschools: The state of reform in Chicago. Chicago:Consortium on Chicago School Research.

Bryk, A. S., Lee, V. E., & Holland, P. B. (1993).Catholic schools and the common good. Cambridge,MA: Harvard University Press.

Bryk, A. S., Lee, V. E., & Smith, J. B. (1990). Highschool organization and its effects on teachers andstudents: An interpretive summary of the research.In W. H. Clune & J. F. Witte (Eds.), Choice and con-trol in American education: VoL 1 (pp. 135-226).New York: Falmer.

Bryk, A. S., Sebring, P. B., Kerbow, D., Rollow, S., &Easton, J. (1998). Charting Chicago school reform.Boulder, CO: Westview Press.

Bryk, A.-S., Thum,Y. M., Easton, J. Q., &Luppescu, S.(1998). Academic productivity of Chicago public ele-mentary schools. Chicago: Consortium on ChicagoSchool Research.

Chicago Public Schools (1995). Childrenfirst: Self-analysis guide. Chicago: Author.

Chubb, J. E., & Moe, T. M. (1990). Politics, markets,and America's schools. Washington, DC: Brook-ings Institution Press.

Cohen, D. K. (1995). What is the system in systemicreform? Educational Researcher, 24(9), 11-17,31.

Cohen, D., & Ball, D. (1996). Reform by the book:What is-or might be-the role of curriculummaterials in teacher learning and instructional re-form? Educational Researcher 25(9), 6-8.

Coleman, J. S., Hoffer, T., & Kilgore, S. (1982). Highschool achievement: Public, Catholic, and privateschools compared. New York: Basic Books.

Consortium for Policy Research in Education. (20015.Bridging the K-12/postsecondary divide with a co-herent K-16 system. University of Pennsylvania:Author. http:/.vww.cpre.org/Puiblications/rb3l.pdf

Consortium on Productivity in the Schools. (1995).Using what we have to get the schools we need: Aproductivity focus for American education. NewYork: Consortium on Productivity in the Schools;Institute on Education and the Economy; TeachersCollege, Columbia University.

Cunningham, P. M., & Allington, R. L. (1999). Class-rooms that work: They can all read and write. 2nd ed.New York: Longman.

Ehnore, R. F. (1996). Getting to scale with good educa-tional practice. Harvard Educational Review 66(1),1-17.

Fuhrman, S. H. (Ed.). (1993). Designing coherenteducational policy: Improving the system. San Fran-cisco: Jossey Bass.

Fuhrman, S. H., & Malen, B. (Eds.). (1991). The pol-itics of curriculum and testing. New York: Falmer.

Greeno,.J. G., Collins, A. M., & Resnick, L. B. (1996).Cognition and learning. In D. C. Berliner &

319

terence


R. Calfee (Eds.), Handbook of Educational Psy-chology (pp. 15-46). New York: Simon SchusterMacmillan.

Hess, F. (1999). Spinning wheels: The politics ofurban school reform. Washington, DC: BrookingsInstitution Press.

Hill, P. T., & Celio, M. B. (1998). Fixing urban schools.Washington, DC: Brookings Institution Press.

Kanfer, R. (1990). Motivation theory and industrialand organizational psychology. In M. D. Dunnette& L. H. Leatta (Eds.), Handbook of industrial andorganizational psychology: VoL 1 (pp. 75-170).2nd ed. Palo Alto, CA: Psychologists Press.

King, M. B., & Newmann, F. M. (1999). School ca-pacity as a goalforprofessional development: Map-ping the terrain in low income schools. Paper pre-sented at the annual conference of the AmericanEducational Research Association, Montreal.

King, M. B., & Newmann, F. M. (2000). Will teacherlearning advance school goals? Phi Delta Kappan,81(8),576-580.

Lawler, E. (1990). High involvement management.San Francisco: Jossey-Bass.

Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In D. C. Berliner & R. Calfee (Eds.),Handbook of educational psychology (pp. 47-62).New York: Simon Schuster Macmillan.

New American Schools Development Corporation.(1991). Designs for a new generation of Americanschools: Requestfor proposal. Roslyn, VA: author.

Newmann, F. M. (1981). Reducing student alienationin high schools: Implications of theory. HarvardEducational Review, 51(4),546-564.

Newmann, F. M., Lopez, G., & Bryk, A. S. (1998).The quality of intellectual work in Chicago schools:A baseline report. Chicago: Consortium on ChicagoSchool Research.

Newmann, F. M., & Sconzert, K. (2000). School im-provement with external partners. Chicago: Con-sortium on Chicago School Research.

Newmann, F. M., & Wehlage, G. G. (1995). Success-ful school restructuring: A report to the public andeducators. Madison: Wisconsin Center on Educa-tion Research.

O'Day, J. A., Goertz, M. E., & Floden, R. E. (1995,December). Building capacity for education reform.CPRE Policy Briefs: Reporting on issues and re-search in education policy (RB-18). New Brunswick,NJ: Rutgers University.

Pallas, A. M., & Neumann, A. (1995). Lost in transla-tion: Applying total quality management to schools,colleges, and universities. In M. T. Hallinam (Ed.),Restructuring schools: Promising practices andpolicies (pp. 31-55). New York: Plenum.

Pittman, T. S. (1998). Motivation. In D. T. Gilbert,S. T. Fiske, & G. Lindsay (Eds). The handbook of

320

social psychology: Vol. 1 (pp. 549-590). 4th ed.New York: McGraw Hill.

Purkey, S. C., & Smith, M. S. (1983). Effectiveschools: A review. The Elementary School Journal,83(4), 427-452.

Raudenbush, S., Bryk, A. S., Cheong, Y. F., & Cong-don, R. (2000). HLM5: Hierarchial Linear andNonlinear Modeling. Lincolnwood, IL: ScientificSoftware.

Rowan, B., & Miskel, C. G. (1999). Institutional the-ory and the study of educational organizations. InK. S. Louis & J. Murphy (Eds.), Handbook of re-search on educational administration (pp.359-384).San Francisco: Jossey Bass.

Scott, W. R., & Meyer, J. W. (1994). Environmentallinkages and organizational complexity: Public andprivate schools. In W. R. Scott and J. W. Meyer(Eds.), Institutional environments and organiza-tions: Structural complexity and individualism(pp. 137-159). Thousand Oaks, CA: Sage.

Sergiovanni, T. J. (1994). Building community inschools. San Francisco: Jossey-Bass.

Slavin, R. E., Madden, N. A., Dolan, L. J., & Wasik,B. A. (1996). Every child, every school: Successforall. Newbury Park, CA: Corwin.

Smith, J. B., Smnith, B. A., and Bryk, A. S. (1998).Setting the pace: Opportunities to learn in Chicago'selementary schools. Chicago: Consortium onChicago School Research.

Smith, M. S., & O'Day, J. A. (1991). Systemic schoolreform. In S. H. Fuhrman &B. Malen (Eds.), The pol-itics of curriculum and testing. New York: Falner.

Smylie, M. A., Bilcer, D. K., Kochanek, J., Sconzert,K., Shipps, D., & Swyers, H. (1998). Getting started:A first look at Chicago Annenberg schools and net-works. Chicago: Consortium on Chicago SchoolResearch, Chicago Annenberg Research Project.

Tyack, D., & Cuban, L. (1995). Tinkering towardutopia. Cambridge, MA: Harvard University Press.

Van der Linden, W., & Hambleton, R. (Eds.). (1997).Handbook of modern item response theory. NewYork: Springer.

Wohlstetter, P., Smyer, R., & Mohrman, S. A. (1994).New boundaries for school-based management: Thehigh involvement model. Educational Evaluationand Policy Analysis 16, 268-286.

Wright, B. D., & Masters, G. N. (1982). Rating scaleanalysis. Chicago: MESA Press.

Authors

FRED M. NEWYMANN is Emeritus Professorof Cunriculum and Instruction at the Universityof Wisconsin, Madison. In recent publications heproposes criteria for authentic achievement andassessment and describes how restructuring in


schools nationwide affects authentic studentachievement, equity, empowerment, profes-sional community, and accountability. He re-cently completed a national study of professionaldevelopment to build capacity in low-incomeschools, and he is participating in a study ofschool reform in Chicago with colleagues in theConsortium on Chicago School Research.

BETSANN SMITH is an Assistant Professorin the Department of Educational Administrationat Michigan State University. She is currentlystudying the design and outcomes of after-schoolprograms for urban school students and explor-ing the relationship between .new accountabilitypolicies and the expansion of students' time inschool.

ELAINE ALLENSWORTH is a Senior Re-search Associate and Interim Associate Directorfor statistical analysis, surveys, and data man-

agement at the Consortium on Chicago SchoolResearch. Her current work focuses on the im-plications of accountability policies for dropoutrates and student progression through school, aswell as the effects of school structure, organiza-tion, and social context on school capacity forimprovement.

ANTHONY S. BRYK is the Marshall FieldIV Professor of Urban Education in the Depart-ment of Sociology at the University of Chicago.He directs the University's Center for School Im-provement and is Senior Director of the Consor-tium on Chicago.School Research. His researchinterests include school organization, urbanschool reform, accountability, and educationalstatistics.

Manuscript received November 6, 2000Revision received June 20, 2001

Accepted September 13, 2001

321

COPYRIGHT INFORMATION

TITLE: Instructional program coherence: what it is and why itshould guide school improvement policy

SOURCE: Educational Evaluation & Policy Analysis 23 no4 Wint2001

WN: 0134903465001

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited. To contact the publisher:http://www.aera.net/.

Copyright 1982-2002 The H.W. Wilson Company. All rights reserved.

instructional program coherence: what it is and why it ... · instructional program coherence...

Documents