inside the black box of school reform: explaining the how ... file52 d. mcdougall et al. wave of...

International Journal of Disability, Development and EducationVol. 54, No. 1, March 2007, pp. 51–89

ISSN 1034-912X (print)/ISSN 1465-346X (online)/07/010051–39© 2007 Taylor & FrancisDOI: 10.1080/10349120601149755

Inside the Black Box of School Reform: Explaining the how and why of change at Getting Results schools

Dennis McDougalla*, William M. Saundersb and Claude GoldenbergcaUniversity of Hawai’i, USA; bPearson Achievement Solutions, USA; cCalifornia State University Long Beach, USATaylor and Francis LtdCIJD_A_214904.sgm10.1080/10349120601149755International Journal of Disability, Development and Education1034-912X (print)/1465-346X (online)Original Article2007Taylor & Francis541000000March [email protected]

This article reports key findings from a process-focused external evaluation that compared a subsetof Getting Results project schools and comparison schools in order to understand the dynamics ofschool-wide reform efforts at these primary schools. Findings shed light on the “black box”of school reform and illuminate the limited empirical basis for understanding the inner workings ofmost reform efforts. We describe how Getting Results Model elements—goals, indicators, assis-tance, leadership, and settings—worked in concert to improve teaching and learning at projectschools. We also describe factors that inhibited and promoted change, as well as implications forhow whole-school reform might be accomplished through purposeful manipulation of these essen-tial change elements.

Keywords: Change; Evaluation; Process; Reform; Schools

Introduction

Whole-school reform proliferated in the United States (U.S.) during the 1990s andreached a high point with passage of Public Law 105-78, commonly know as “Obey-Porter”, in 1997. The bill authorised the Comprehensive School Reform Demon-stration Program and triggered hundreds of whole-school reform efforts nationwide.Many promising approaches to whole-school reform have been developed over thepast 15 years (American Institutes for Research, 1999; Borman, Hewes, Overman,& Brown, 2003; Desimone, 2002; Slavin & Fashola, 1998). Even before this recent

*Corresponding author. University of Hawai’i, Department of Special Education, 124 Wist Hall,1776 University Avenue, Honolulu, HI 96822, USA. Email: [email protected]

52 D. McDougall et al.

wave of whole-school reform models, educators have for years attempted to findways to improve entire schools (see, e.g., the “effective schools” research of the1970s and 1980s; Bliss, Fireston, & Richards, 1991). However, the empirical basisfor understanding the actual process of school reform is very limited. Few studieshave examined the effects of reform models within experimental or quasi-experi-mental research designs that would permit clear conclusions about the effects ofreform models on student outcomes. Even fewer studies have looked directly at theprocess of reform to examine prospectively the dynamics leading to school improve-ment. Fullan (2000) reviewed research on schools with effective collaborative schoolcultures and noted, “We know nothing about how these particular schools got thatway, let alone how to go about producing more of them” (p. 582). Although theliterature systematically examining the processes and outcomes of school improve-ment efforts has grown enormously over the past decade (e.g., Berends, Kirby,Naftel, & McKelvey, 2001; Borman et al., 2003; Desimone, 2002), the problemidentified by Sarason (1972) 35 years ago is still largely with us: We have very littledirect observational data to document how schools change from being less to moreeffective in educating their students. The problem is significant because withoutdetailed knowledge of how schools change, we know little about why school reformefforts obtain the results they do (Desimone). This is the “black box” problem weaddress in this article.

The study we report here contributes to the literature on school change for diverseand traditionally low-achieving populations of students. We report findings from aqualitative, prospective, external evaluation (McDougall, 2002) that illuminated theinternal workings of a whole-school reform project. The Getting Results (GR) Modelthat guides this school change project is comprised of five interdependentelements—goals, indicators, assistance, leadership, and settings. The evaluationstudy answers a set of five questions about model implementation, processes, andoutcomes. We discuss findings in relation to school reform and the importance ofunderstanding its inner workings.

Background and Basis of the GR School Change Model

In a previous project, we developed a school change model that was instrumental inproducing substantial changes in teaching and learning at one pilot site—a primaryschool that served primarily Latino children and families, in Southern California(Goldenberg, 2004; Goldenberg, Saunders, & Gallimore, 1996; Goldenberg &Sullivan, 1994). We refer to this model as the School Change/Getting Results Model,or, more simply, the GR Model. Over a 6-year period, the pilot school shifted frombeing the lowest achieving school in the school district to surpassing district averageson both standardised tests and performance-based assessments. Fashola, Slavin,Calderón, and Durán (1996) identified the GR Model as one of only three schoolchange models with demonstrated effectiveness in majority Latino schools.

The GR Model utilises five elements to leverage changes in educators’ instructionalbehaviours and attitudes, and student outcomes. These elements include: (a) goals

Inside the Black Box of School Reform 53

that are set, shared, and articulated explicitly by the school community; (b) meaning-ful indicators that measure progress toward goal attainment; (c) assistance by capableothers from within and outside the school; and (d) focused leadership that supportsand pressures goal attainment. “Settings” is a fifth element and a super-ordinateconcept in the model. Implementing the GR Model requires establishing new settingsand reformulating existing settings for educators to meet, collaborate, receive assis-tance, and do important, instructional tasks (Sarason, 1972; Tharp & Gallimore,1988). Within the context of these settings, the other change elements work inconcert to improve teaching, learning, and achievement in any targeted curriculumor subject. In working with schools, we discovered a need to focus specifically andconsistently on creating and sustaining concrete settings, at each school, wherechange elements are operationalised in effective ways—ways that permit educators todo the very important work of improving teaching and learning.

Research provides strong support for GR Model elements. Classic, and morerecent, educational research documents the importance of explicit academic goals(Carter & Chatfield, 1986; Peterson & Lezotte, 1991). Goals are important forachieving long-term change and substantive improvements because they are vital formaintaining a coherent and stable student-centered vision. Indicators that measureprogress toward agreed-upon goals reinforce the importance of the learning goals andhelp teachers and administrators gauge their goal-directed efforts. Consistent use ofachievement indicators is related to improvements in student outcomes (Brophy &Good, 1986; Edmonds, 1979). Assistance from fellow professionals (Lieberman,1988a, b; Little, 1982; Rosenholtz, 1991), including training from consultants (Joyce& Showers, 1983), is essential for successful reform (Saunders et al., 2001). Success-ful professional development models go well beyond “one-shot” workshops andpresentations that attempt to train teachers in short order (Goldenberg & Gallimore,1991). Professional development must be seen as long-term assistance to help educa-tors acquire knowledge and skills essential for accomplishing agreed-upon goals(Darling-Hammond, 1997). Finally, leadership must both support and pressurechanges necessary for goal attainment (Fullan, 1993). These two leadership dimen-sions complement each other, producing a creative tension that is perhaps the mostelusive but important aspect of leadership (Blase, 1987; Bliss et al., 1991; Miles,1983). Effective school leadership has long been recognised as fundamental to creat-ing more effective schools, regardless of the cultural or linguistic background ofstudents (August & Hakuta, 1997). The GR Model treats all five elements, includingleadership, as highly interdependent. Strong leadership artfully combines pressureand support in a way that moves schools relentlessly toward accomplishing studentachievement goals, utilising indicators, cultivating assistance and collaboration, andbuilding productive school settings.

Current “Scale-up” Phase of the GR Project

The most current GR Model project “scaled up” our previous school reform effortsand achieved simultaneously, at nine schools, improvements in teaching and


learning similar to those achieved at the original pilot school. We examined prospec-tively a model of school change within a quasi-experimental research design. Fifteenelementary schools participated during this scale-up phase—nine schools that usedthe GR Model, all of whom entered the study and initiated GR Model implementa-tion voluntarily, and six demographically similar schools in the same district thatagreed to be comparison sites. At the study’s inception, GR schools and comparisonschools, as two groups, had nearly identical mean achievement scores on annual,statewide, standardised tests. These K-5 schools served predominantly poor andworking class Hispanic communities. GR replication sites utilised modest localfunding—US$100,000 per school over a 3-year period—to support implementation.A research grant supported GR staff’s assistance to schools, as well as data collectionand analyses.

As members of the same, large, urban school district, all schools (both GR andcomparison) functioned under the same reform umbrella set forth by Californiastate legislation and local school district mandates. Four major reforms wereunderway in this local school district when the scale-up project began in 1997: (a)class size reduction (20 students to one teacher) at grades K-3; (b) state-established and district-implemented content standards for language arts andmathematics; (c) annual achievement accountabilities and reporting based on state-mandated, standardised tests; and (d) common curriculum, mandated training,and school-based coaches for reading/language arts and mathematics. Primaryschools in this district showed steady increases in student achievement since thelate 1990s, very probably related to these major district and state reform efforts.Our quantitative evaluation of achievement impact (Saunders, 2003), focused onthe additive effects of the GR Model, and the extent to which outcomes andincreases over time at GR schools surpassed those of other, comparable, schools inthis large, urban district.

Putting GR Model elements into practice. In brief, GR Model implementation estab-lished several settings and processes designed to ensure effective application ofmodel elements—goals, indicators, assistance, and leadership. Settings includedAcademic Achievement Leadership Teams (AALTs or ALTs), Grade Level Team(GLT) meetings, and GR Principals’ meetings. GR also established beginning,middle, and end-of-year (BME) assessments in reading, writing, and oral languageproficiency. GR staff provided on-site assistance to support school efforts to estab-lish and maintain these settings. GR staff also provided annual, 3-day, beginning-of-year leadership training institutes for school teams, and a 1-day, mid-year follow-up.GR staff spent approximately one day every two weeks at each GR school, metmonthly with principals, and facilitated many of the aforementioned GR settings.Schools also used materials developed and tested by GR staff, including rubrics andchecklists to evaluate GR implementation; beginning and advanced training manu-als for ALTs; a series of three videotapes that described the GR Model and illus-trated various settings and processes; reading, writing, and oral proficiency


assessments for schools that did not already have such assessments; worksheets andguidelines for analysing achievement data and student work samples; and guidelinesand training modules for GR coaches.

Academic achievement at GR and comparison schools. During this scale-up phase, GRschools (N = 9) showed significantly greater gains in academic achievement thancomparison schools (N = 6) across 5 years. The following data are based on testscores averaged across Grades 2–5 and across reading, mathematics, language, andspelling subtests on the Stanford Achievement Test, 9th edition (Stanford 9;Harcourt Educational Measurement, 1996). Mean achievement for GR schools, inNormal Curve Equivalent (NCE) units, increased from 32.8 (SD = 4.8) in 1997 to48.3 (SD = 4.2) in 2002, a net increase of 15.5 NCE units. Corresponding NationalPercentile Ranks increased from 21st to 47th. In contrast, NCE means for the sixcomparison schools increased from 32.0 (SD = 6.3) in 1997 to 43.0 (SD = 6.0) in2002, a net increase of 11 NCE units. Corresponding National Percentile Ranksincreased from 20th to 37th. Whereas achievement, as measured by NCE means,was nearly identical at GR and comparison schools in 1997 (32.8 versus 32.0), by2002, achievement at GR schools exceeded that of comparable schools by more than5 NCEs (48.3 versus 43.0)—an adjusted effect size of 0.75. An effect size of 0.75, inthis case, means that GR schools scored, on average, 0.75 SD units higher than theaverage achieved by schools in the comparison group. This magnitude of effect isconsidered in the high-moderate to large range (Cohen, 1988), and it compares veryfavorably with effect sizes of other school reform efforts, most of which are below0.40 (Borman et al., 2003).

Achievement also increased more rapidly for GR schools than for the total popula-tion of elementary schools (N = 600+) in the district. Mean NCEs district-wide forGrades 2–5 increased from 36.5 in 1997 (nearly four points higher than the GRschools) to 47.3 in 2002 (one point lower than GR schools). In sum, GR schoolsincreased an average of 15.5 points, six comparison schools increased by an averageof 11.0 points, and schools district-wide increased by an average of 10.8 points. SeeSaunders (2003) for a more comprehensive and technical presentation of achieve-ment results.

Methods

GR project leaders hired an external evaluator who conducted an independent, on-site, process evaluation, throughout the 2001–02 academic year (McDougall, 2002).The evaluator investigated qualitatively the how and why of school change processesand achievement gains at seven, purposefully selected, demographically similar,case-study schools—four of nine GR schools plus three of six comparison schools(see Table 1). The evaluator used qualitative methods, a comparative case-studyresearch design, and rubric-based coding and ratings to answer the followingresearch questions (RQs):


● RQ 1: To what extent is the GR Model implemented in GR project schools?● RQ 2: To what extent does the GR Model establish processes discernable at GR,

but not comparison, schools?● RQ 3: What does implementation of the GR Model do to impact student

achievement?● RQ 4: What helps and hinders schools’ implementation of the GR Model?● RQ 5: What has changed, as a result of GR Model implementation, from the

perspectives of participants and the external evaluator?

Data Collection and Analyses

Data collection included audio-taped and transcribed teacher focus groups and prin-cipal interviews; observations and field notes of GLT meetings, ALT meetings,school-wide faculty meetings, professional development sessions, and principals’meetings; and document retrieval. The evaluator collected additional informationfrom spontaneous contacts with participants. In addition, the evaluator collectedfunctionally similar types and amounts of information from each case-study schoolby observing and participating in over 100 events, in formal and informal settings, atGR and comparison schools.

Data analyses included qualitative analysis, plus GR Model, rubric-based, codingand rating of data from the aforementioned sources. The evaluator analysed dataduring and after a 9-month data collection period, and used: (a) the constantcomparative method to formulate and refine findings (Glaser & Strauss, 1967); (b)triangulation to corroborate findings from multiple data sources, across individuals,time, and settings (Denzin, 1978; Miles & Huberman, 1994); and (c) memberchecks whereby participants provided feedback on emerging findings (Goffman,1959; Kvale, 1996; Lincoln & Guba, 1985; Manning, 1997; Taylor & Bogden,1998). The evaluator also used a transformative strategy (Creswell, 2003) to codeand rate data.

Coding and rating episodes. The evaluator used the GR Model Rubric (Table 2) andmore detailed operational charts to code and rate episodes from all data sources.Episodes were narrative data that contained information pertaining to GR elements.Episodes ranged in length from a single sentence to multiple paragraphs. The evalu-ator used applicable elements and corresponding descriptive criteria of the GRModel, in Table 2, to code all episodes. For example, episodes coded L-A containedinformation that pertained to descriptive criterion A (i.e., leadership’s consistency infocusing on school-wide, academic achievement goals) of the leadership (L)element. Next, the evaluator rated the coded episodes 4, 3, 2, or 1 to indicate therubric level that applied to the information contained in the episode. Some episodesincluded information that pertained to adjacent rubric levels. Such episodes required“in-between” ratings, such as 2 to 3. Thus, the evaluator used seven rubric-basedoptions to rate episodes: 1, 1 to 2, 2, 2 to 3, 3, 3 to 4, and 4. Initial coding and rating

Inside the Black Box of School Reform 57T

able

1.

Dem

ogra

phic

and

des

crip

tive

info

rmat

ion

for

case

-stu

dy s

choo

ls

GR

Pro

ject

sch

ools

(N

= 4

)C

ompa

riso

n sc

hool

s (N

= 3

)

Oak

Pin

eE

lmF

irK

amH

all

Lot

Sch

edul

eY

ear-

roun

d m

ulti

-tra

ckT

radi

tion

al

sing

le-t

rack

Yea

r-ro

und

mul

ti-t

rack

Yea

r-ro

und

mul

ti-t

rack

Yea

r-ro

und

mul

ti-t

rack

Yea

r-ro

und

mul

ti-t

rack

Yea

r-ro

und

mul

ti-t

rack

Stu

dent

s en

rolle

d1,

204

557

1,38

81,

211

1,25

91,

123

1,24

9T

itle

I s

choo

laY

esY

esY

esY

esY

esY

esY

esS

tude

nts/

clas

s (M

)20

.717

.421

.420

.521

.421

.420

.8S

tude

nts/

com

pute

r55

1015

1411

2357

Cla

sses

on

Inte

rnet

00

02

886

0E

nglis

h L

earn

erb

(%)

7944

7276

6172

76L

angu

ages

of

EL

S

= 7

7S

= 3

5S

= 6

6S

= 6

9S

= 6

1S

= 7

2S

= 7

6S

tude

ntsc

(%)

Ar

= 0

.6A

r =

5K

h =

3A

r =

6A

ll ot

her

= 0

P =

0.1

All

othe

r =

0P

= 0

.5R

= 1

Ar

= 2

Ko

= 0

.3A

r =

0.1

F =

0.2

Ko

= 1

T =

0.5

T =

0.2

Fre

e/re

duce

d m

eals

(%

)95

7392

9793

9598

Cal

Wor

ksd

(%)

2219

2926

1316

13C

ompe

nsat

ory

Ed.

e (%

)91

104

9997

9597

98S

tude

nt e

thni

city

f (%

)H

= 9

1H

= 5

5H

= 8

3 H

= 8

4H

= 9

8H

= 9

8H

= 9

9W

= 4

W =

27

W =

5

Af

= 5

W =

11

W =

2W

= 1

Af

= 0

.2

Af

= 3

Af

= 1

4A

s =

5A

f =

3F

= 0

.2A

f =

0.5

W =

0.1

58 D. McDougall et al.T

able

1.

(Con

tinue

d)

GR

Pro

ject

sch

ools

(N

= 4

)C

ompa

riso

n sc

hool

s (N

= 3

)

Oak

Pin

eE

lmF

irK

amH

all

Lot

As

= 1

A

s =

2A

s =

1A

s =

0.1

As

= 0

.1A

s =

0.3

F =

0.1

Tea

cher

eth

nici

tyf (

%)

W =

52

W =

63

W =

57

W =

62

H =

60

H =

62

H =

45

H =

41

H =

20

H =

30

H =

25

W =

25

W =

28

W =

39

Af

= 5

Af

= 9

Af

= 7

As

= 6

F =

5A

f =

5

As

= 1

1A

s =

2A

s =

9A

s =

1A

f =

3A

s, A

f =

3F

= 3

Af

= 3

Tea

cher

s w

/o f

ull c

rede

ntia

lg26

/57

= 4

6%13

/33

= 3

9%18

/67

= 2

7%19

/63

= 3

0%22

/62

= 3

5%19

/56

= 3

4%25

/65

= 3

1%

Not

es: P

erce

ntag

e ca

lcul

ated

usi

ng t

he t

otal

num

ber

of s

tude

nts

enro

lled

(or

teac

hers

em

ploy

ed)

at t

he s

choo

l, ro

unde

d to

the

nea

rest

who

le

num

ber

unle

ss o

ther

wis

e in

dica

ted.

a Qua

lifies

for

fede

ral f

unds

bas

ed o

n a

high

-pov

erty

stu

dent

bod

y. b “

Eng

lish

Lea

rner

s” (

EL

) (i

.e.,

form

er t

erm

used

was

“L

EP

” or

stu

dent

s w

ith

limit

ed E

nglis

h pr

ofici

ency

). c L

angu

age:

Ar

= A

rmen

ian,

F =

Far

si, K

h =

Khm

er, K

o =

Kor

ean,

P =

Pun

jabi

, R

= R

ussi

an, S

= S

pani

sh, T

= T

agal

og. d C

alW

orks

= C

alif

orni

a W

ork

Opp

ortu

nity

and

Res

pons

ibili

ty t

o K

ids;

for

mer

ly A

id t

o F

amili

es w

ith

Dep

ende

nt C

hild

ren.

e Pro

gram

me

for

educ

atio

nally

dis

adva

ntag

ed s

tude

nts.

f Eth

nici

ty: A

f =

Afr

ican

Am

eric

an, A

s =

Asi

an, F

= F

ilipi

no, H

=

His

pani

c, W

= W

hite

. g Wit

hout

(w

/o)

full

cred

enti

al =

em

erge

ncy

cred

enti

al/h

ire,

wai

ver,

pre

-int

ern,

Dis

tric

t in

tern

, Uni

vers

ity

inte

rn; D

ata

repo

rted

or

deri

ved

from

ww

w.e

d-da

ta.k

12.c

a.us

/sch

ool/s

choo

lpro

file;

199

9–20

00 =

mos

t re

cent

ava

ilabl

e da

ta d

urin

g ev

alua

tion

stu

dy.


able

2.

GR

Mod

el R

ubri

c

Ele

men

tsL

evel

1L

evel

2L

evel

3L

evel

4

Goa

ls th

at a

re s

et

and

shar

edN

o sc

hool

-wid

e ac

adem

ic

achi

evem

ent

goal

s ar

e id

enti

fied

.

Sch

ool-

wid

e ac

adem

ic

achi

evem

ent

goal

s id

enti

fied

; go

als

are

fair

ly g

ener

al.

Sch

ool-

wid

e ac

adem

ic

achi

evem

ent

goal

s id

enti

fied

; goa

ls a

re

spec

ific

and

con

cret

e.

Sch

ool-

wid

e ac

adem

ic

achi

evem

ent

goal

s id

enti

fied

; go

als

are

spec

ific

, con

cret

e,

and

publ

ishe

d. A

All/

near

ly a

ll te

ache

rs a

re

unaw

are

of g

oals

or

assu

me

goal

s ar

e un

rela

ted

to

teac

hing

.

Mos

t te

ache

rs m

ay b

e un

awar

e of

the

goa

ls o

r as

sum

e go

als

are

unre

late

d to

th

eir

teac

hing

.

Mos

t tea

cher

s ar

e aw

are

of

the

goal

s an

d ne

w te

ache

rs

are

usua

lly t

old

abou

t go

als.

Tea

cher

s th

ink

the

goal

s ar

e im

port

ant;

new

tea

cher

s ar

e ex

plic

itly

intr

oduc

ed t

o th

e go

als.

BT

each

ers

did

not

play

som

e ro

le in

iden

tify

ing

goal

s.F

ew t

each

ers

(and

oth

ers)

pl

ayed

som

e ro

le in

id

enti

fyin

g go

als.

Tea

cher

s (a

nd o

ther

s)

play

ed s

ome

role

in

iden

tify

ing

goal

s.

Tea

cher

s (a

nd o

ther

s) h

elpe

d de

velo

p or

ada

pt t

he g

oals

. C

Indi

vidu

al t

each

ers

do n

ot

use

the

goal

s in

pla

nnin

g in

stru

ctio

n.

Few

indi

vidu

al t

each

ers

use

the

goal

s in

pla

nnin

g in

stru

ctio

n.

Som

e in

divi

dual

tea

cher

s us

e th

e go

als

in p

lann

ing

inst

ruct

ion.

Tea

cher

s us

e th

e go

als

indi

vidu

ally

and

col

lect

ivel

y in

pl

anni

ng in

stru

ctio

n. D

Indi

cato

rs th

at

mea

sure

suc

cess

No

scho

ol-w

ide

indi

cato

rs o

f st

uden

t ac

hiev

emen

t ar

e us

ed.

Som

e sc

hool

-wid

e in

dica

tors

us

ed t

o ex

amin

e ge

nera

l tr

ends

(up

/dow

n) in

stu

dent

ac

hiev

emen

t.

Spe

cifi

c sc

hool

-wid

e in

dica

tors

use

d to

ass

ess

whe

ther

stu

dent

s ar

e m

eeti

ng id

enti

fied

goa

ls.

Spe

cifi

c sc

hool

-wid

e in

dica

tors

use

d to

ass

ess

whe

ther

stu

dent

s ar

e m

eeti

ng

iden

tifi

ed g

oals

. AM

anda

ted

test

s ar

e gi

ven,

but

no

t us

ed t

o re

view

in

form

atio

n ab

out

stud

ent

achi

evem

ent.

Mos

t te

ache

rs m

ay b

e un

awar

e of

indi

cato

rs o

r do

no

t vi

ew t

hem

as

valid

and

us

eful

sou

rces

of

info

rmat

ion.

Mos

t tea

cher

s ar

e aw

are

of

the

indi

cato

rs a

nd th

ey a

re

som

ewha

t in

tere

sted

in

the

info

rmat

ion.

Mos

t te

ache

rs t

hink

the

in

dica

tors

are

impo

rtan

t an

d th

ey v

iew

the

info

rmat

ion

as

valid

and

use

ful.

BN

one/

near

ly n

o in

divi

dual

te

ache

rs u

se in

form

atio

n fr

om

indi

cato

rs t

o re

fine

the

ir

inst

ruct

ion

and

curr

icul

um.

Few

indi

vidu

al t

each

ers

use

info

rmat

ion

from

indi

cato

rs

to r

efin

e th

eir

inst

ruct

ion

and

curr

icul

um.

Som

e in

divi

dual

tea

cher

s us

e th

e in

form

atio

n to

re

fine

thei

r ins

truc

tion

and

cu

rric

ulum

.

Mos

t te

ache

rs u

se t

he

indi

cato

rs in

divi

dual

ly a

nd

colle

ctiv

ely

to r

efin

e in

stru

ctio

n an

d cu

rric

ulum

. C


able

2.

(Con

tinue

d)

Ele

men

tsL

evel

1L

evel

2L

evel

3L

evel

4

Ass

ista

nce

from

ca

pabl

e ot

hers

No/

very

few

opp

ortu

niti

es fo

r te

ache

rs t

o re

ceiv

e as

sist

ance

.In

term

itte

nt o

ppor

tuni

ties

for

assi

stan

ce.

Reg

ular

, ong

oing

, sch

ool-

wid

e op

port

unit

ies

for

assi

stan

ce.

Reg

ular

, ong

oing

, sch

ool-

wid

e op

port

unit

ies

for

assi

stan

ce. A

Pro

vide

d by

som

eone

who

do

es n

ot h

ave

rele

vant

ex

peri

ence

, exp

erti

se, r

espe

ct.

Pro

vide

d by

som

eone

who

m

ay o

r m

ay n

ot h

ave

rele

vant

ex

peri

ence

and

exp

erti

se.

Pro

vide

d by

som

eone

wit

h re

leva

nt e

xper

ienc

e an

d ex

pert

ise.

Pro

vide

d by

som

eone

wit

h re

leva

nt e

xper

ienc

e an

d ex

pert

ise

who

has

or

gain

s th

e re

spec

t of

mos

t te

ache

rs. B

Unr

elat

ed t

o m

eeti

ng s

choo

l-w

ide

achi

evem

ent

goal

s.M

ostl

y un

rela

ted

to m

eeti

ng

scho

ol-w

ide

achi

evem

ent

goal

s.

Rel

ated

to

mee

ting

id

enti

fied

sch

ool-

wid

e ac

hiev

emen

t go

als.

Foc

used

dir

ectl

y on

mee

ting

id

enti

fied

sch

ool-

wid

e ac

hiev

emen

t go

als.

CU

nava

ilabl

e/av

aila

ble

brie

fly.

Ava

ilabl

e ov

er s

hort

tim

e pe

riod

.A

vaila

ble

over

ext

ende

d pe

riod

of

tim

e.A

vaila

ble

over

ext

ende

d pe

riod

of

tim

e. D

Lea

ders

hip

that

su

ppor

ts a

nd

pres

sure

s

Lea

ders

hip

ofte

n di

stra

cted

or

doe

s no

t fo

cus

on

acad

emic

ach

ieve

men

t.

Lea

ders

hip

gene

rally

foc

uses

on

aca

dem

ic a

chie

vem

ent

scho

ol-w

ide.

Lea

ders

hip

focu

ses

on

mee

ting

iden

tifi

ed

achi

evem

ent

goal

s sc

hool

-w

ide.

Lea

ders

hip

focu

ses

cons

iste

ntly

on

mee

ting

id

enti

fied

ach

. goa

ls s

choo

l-w

ide.

AD

oes

not

part

icip

ate

in

esta

blis

hing

and

rev

isin

g pl

ans.

Iden

tifi

es g

ener

al a

reas

of

need

, rec

omm

ends

po

ssib

iliti

es.

Hel

ps e

stab

lish

a co

ncre

te

plan

to

achi

eve

goal

s.H

elps

est

ablis

h co

ncre

te p

lan,

an

d ov

er t

ime

cont

inue

s to

id

enti

fy n

ext

step

s. B

Pro

vide

s lit

tle

if a

ny

enco

urag

emen

t an

d re

sour

ces.

Pro

vide

s so

me

enco

urag

emen

t an

d av

aila

ble

reso

urce

s.

Pro

vide

s en

cour

agem

ent

and

need

ed r

esou

rces

.P

rovi

des

ampl

e en

cour

agem

ent

and

reso

urce

s. C

Rar

ely

if e

ver

enco

urag

es

indi

vidu

al a

nd c

olle

ctiv

e re

spon

sibi

lity

Usu

ally

doe

s no

t en

cour

age

indi

vidu

al a

nd c

olle

ctiv

e re

spon

sibi

lity.

Enc

oura

ges

indi

vidu

al a

nd

colle

ctiv

e re

spon

sibi

lity.

Cla

rifi

es a

nd h

olds

peo

ple

acco

unta

ble

for

thei

r in

divi

dual

and

col

lect

ive

resp

onsi

bilit

ies.

D


able

2.

(Con

tinue

d)

Ele

men

tsL

evel

1L

evel

2L

evel

3L

evel

4

Set

ting

s th

at a

llow

st

aff t

o ge

t im

port

ant t

hing

s do

ne

Tim

e ou

tsid

e of

cla

ssro

om is

us

ed to

cla

rify

pro

cedu

res

and

polic

ies,

and

del

iver

req

uire

d tr

aini

ngs.

Tim

e ou

tsid

e of

the

cl

assr

oom

is r

egul

arly

use

d to

de

liver

nec

essa

ry in

form

atio

n,

and

disc

uss

acad

emic

ac

hiev

emen

ts.

Tim

e ou

tsid

e of

the

cl

assr

oom

is r

egul

arly

sc

hedu

led

and

gene

rally

us

ed t

o w

ork

on t

hing

s re

late

d to

ach

ievi

ng

acad

emic

goa

ls.

Tim

e ou

tsid

e of

the

clas

sroo

m

is d

epen

dabl

y sc

hedu

led

and

cons

iste

ntly

use

d to

get

don

e im

port

ant

thin

gs r

elat

ed t

o ac

hiev

ing

acad

emic

goa

ls. A

Tea

cher

s an

d ad

min

istr

ator

s vi

ew m

eeti

ngs

as a

n in

trus

ion

on t

heir

tim

e.

Mos

t te

ache

rs/a

dmin

istr

ator

s vi

ew m

eeti

ngs

as p

art

of t

heir

w

ork,

but

do

not

valu

e it

as

impo

rtan

t.

Mos

t te

ache

rs a

nd

adm

inis

trat

ors

valu

e ti

me

to m

eet

and

colla

bora

te a

s a

wor

thw

hile

use

of

tim

e.

Tea

cher

s an

d ad

min

istr

ator

s va

lue

tim

e to

mee

t,

colla

bora

te a

nd g

et t

hing

s do

ne a

s an

impo

rtan

t pa

rt o

f th

eir

wor

k. B

Mee

ting

s ar

e ra

rely

eff

ecti

ve

and

alm

ost

neve

r ef

fici

ent.

Som

e m

eeti

ngs

are

effi

cien

t an

d ef

fect

ive.

Mos

t m

eeti

ngs

are

sche

dule

d, p

lann

ed, a

nd

carr

ied

out

effi

cien

tly

and

effe

ctiv

ely.

Nea

rly

all m

eeti

ngs

cons

iste

ntly

sch

edul

ed,

plan

ned,

and

car

ried

out

ef

fici

entl

y an

d ef

fect

ivel

y. C

Not

e: B

old

lett

ers

A, B

, C, a

nd D

iden

tify

des

crip

tive

cri

teri

a (s

can

hori

zont

ally

) fo

r ea

ch e

lem

ent

of t

he r

ubri

c.


of all data sources produced 2,940 episodes. The evaluator subsequently reviewedall episodes, eliminated episodes that lacked sufficient content or clarity, andidentified episodes that were overlooked during the initial coding and rating. Thissystematic review resulted in a net reduction of 309 episodes for a final total of 2,631coded-rated episodes.

Assigning ratings to GR elements and schools. As was the case for individual episodes,the evaluator utilised seven rubric-based options to rate GR elements: 1, 1 to 2, 2, 2to 3, 3, 3 to 4, and 4. The evaluator assigned element ratings, for each school, basedon the mean and median ratings for that element’s distribution of episode ratings(see Table 3). The evaluator assigned a single rating to an element if its episoderating distribution had a similar mean and median. However, if these measures ofcentral tendency were not consistent (e.g., if the mean suggested a rating of 3 andthe median suggested a rating of 3 to 4), then the evaluator assigned a dual rating forthe element. The evaluator also assigned an overall rating of school change, to eachschool, based on the pattern of element ratings for each of the five GR Modelelements. The bold numbers in Table 3 highlights rating patterns.

Reliability of episode ratings. Secondary evaluators conducted reliability checks ofrubric-based episode ratings. Overall inter-rater agreement for exact agreementsequaled 88%, for 104 episodes selected at random from 2,631 total episodes, usingthe formula “agreements divided by agreements plus disagreements multiplied by100%”. Overall inter-rater agreement for exact plus adjacent agreements equaled100% using the same formula.

Results

RQ 1: To what extent is the GR Model implemented at GR schools?

GR Model implementation was fairly strong at three of the four GR schools, includ-ing Oak, Pine, and Elm, with overall school ratings of 3 to 4, and comparativelyweaker at Fir, which was rated 2 to 3. As seen in Table 3, ratings for each of the fiveGR elements at GR schools varied somewhat between and within schools. In addi-tion, ratings for the settings element, based on direct observations of GLT meetingsat GR schools, indicated that K-3 teachers tended to implement GR proceduresmore effectively than their colleagues in Grades 4 and 5.

RQ 2: To what extent does the GR Model establish processes observable at GR schools but not at comparison schools?

Finding 1. GR schools utilised processes associated with the five elements of the GRModel more frequently and more effectively than comparison schools. Ratings for thefive change elements at GR schools, except Fir, exceeded ratings for the comparison


Table 3. Frequency distribution of episode ratings for GR elements at GR and comparison schools (ratings based on GR Model Rubric levels)

Rating

School Element 1 1 to 2 2 2 to 3 3 3 to 4 4

Oak (GR) Goals 1 2 9 5 11 4 15Indicators 0 2 5 0 9 33 20Assistance 1 0 17 21 14 100 101Leadership 4 11 2 18 16 45 60Settings 0 1 0 6 7 16 6

Pine (GR) Goals 0 0 4 8 12 56 22Indicators 2 4 8 9 32 29 13Assistance 0 3 7 5 26 61 18Leadership 0 12 7 22 27 56 41Settings 0 0 0 3 4 14 9

Elm (GR) Goals 1 0 3 5 8 18 16Indicators 2 7 2 4 1 11 16Assistance 0 2 4 6 21 39 24Leadership 15 24 6 13 17 5 12Settings 0 2 0 10 6 20 3

Fir (GR) Goals 2 3 12 21 16 0 2Indicators 0 7 12 14 15 1 3Assistance 0 31 12 31 15 21 1Leadership 13 39 28 33 16 8 0Settings 1 2 3 14 7 3 0

Kam (Comparison) Goals 0 9 4 26 17 15 0Indicators 1 6 4 11 14 21 0Assistance 2 3 11 23 24 7 1Leadership 1 14 6 1 18 32 5Settings 0 11 2 13 5 2 0

Hall (Comparison) Goals 0 30 17 10 9 1 0Indicators 12 4 5 10 10 4 0Assistance 8 9 11 30 19 0 0Leadership 24 19 5 2 24 17 3Settings 0 19 6 8 0 0 0

Lot (Comparison) Goals 1 16 5 3 11 6 0Indicators 9 13 1 7 10 8 1Assistance 0 6 13 10 11 0 1Leadership 1 22 0 8 11 17 7Settings 1 2 2 13 3 1 0

Notes: Bold numbers indicate the element rating based on mean and median values for distribution of episode ratings for that element. One bold cell per row indicates the evaluator applied a single rating for the element based on similar mean and median values for that distribution of episode ratings. Two bold cells per row indicate the evaluator applied a dual rating for the element based on dissimilar mean and median values for that distribution of episode ratings


schools (see Table 3). Likewise, the overall school rating for each GR school (3 to 4)except Fir (2 to 3) exceeded that of comparison schools Kam and Lot (2 to 3) andHall (2). With few exceptions, GLTs at GR schools used more effective processes toconduct meetings than GLTs at comparison schools. Comparative ratings for thesettings element of the GR Model indicated that teachers at GR schools accom-plished important instructional tasks more frequently and more effectively, duringtheir GLT meetings, than teachers at comparisons schools (see Table 4).

Qualitative analyses indicated that teachers’ behaviours and discourse, as well asmeeting procedures and outcomes, differed appreciably during most team meetingsat GR versus comparison schools (see Table 5). Notable examples, all suggestingmore positive environments existed in GR team meetings, included: (a) more consis-tent focus, planning, and time for academic topics, goals, and indicators and lesstime discussing topics or doing tasks of a non-academic nature, such as duplicatingor collating materials, and planning field trips; (b) analysis of students’ productsabove and beyond state or district-mandated assessments (e.g., BME writing assess-ments at GR schools versus district-mandated, once-per-year, writing assessments atcomparison schools); (c) discussing the relation between instruction, studentoutcomes, and the need for instructional changes; (d) modeling instructional meth-ods for colleagues; (e) assigning and completing goal-related assignments, and usingacademic data with follow-up at subsequent meetings; (f) preparing and evaluatingmutually agreed upon teaching strategies; (g) teachers’ consistent versus sporadicattendance and participation at the meetings; (h) teachers’ punctual arrivals anddepartures versus late arrivals and early departures; (i) principals’ participationversus non-attendance at meetings; (j) teachers’ use of typed agendas and priorawareness of meeting topics versus no agenda and limited, or last-minute, awareness

Table 4. Ratings for the setting element for grade-level team meetings at GR and comparison schools

Rubric rating

1 1 to 2 2 2 to 3 3 3 to 4 4

GR schools FFFFFFFF FF FFF FFPPPEE PPPOO EEEEEE PPPOOO

EOOOO OO(1) (2) (3) (20) (7) (6) (6)

Comparison schools HHHHH HHKKKHHHHH HKL KKKKL KKLHKKLL LLLLLL

L(15) (3) (17) (3) (0) (0)

Note: Each letter represents one episode rating, for one school, for one descriptive criterion from the settings element of the GR Model Rubric. Numbers in parentheses indicate the total number of settings ratings assigned at rubric level. Schools: E = Elm, F = Fir, H = Hall, K = Kam, L = Lot, O = Oak, P = Pine.


able

5.

Gra

de-l

evel

tea

m m

eeti

ngs

at G

R s

choo

ls v

ersu

s co

mpa

riso

n sc

hool

s

GR

sch

ools

Com

pari

son

scho

ols

TC

init

iate

s/te

lls o

ther

tea

cher

s to

pul

l out

Ope

n C

ourt

re

adin

g bo

ok fr

om t

he s

tude

nt d

esks

at

whi

ch t

hey

are

sitt

ing,

op

en t

o st

ory,

“M

ake

Way

for

Duc

klin

gs”.

Pag

e 17

6 is

fir

st

page

of s

tory

. In

desc

ribi

ng t

o ot

her

teac

hers

how

she

del

iver

s in

stru

ctio

n on

the

ski

ll of

new

par

agra

phs,

TC

ask

s th

e te

ache

rs, a

nd s

ays

that

she

ask

s/te

ache

s he

r st

uden

ts t

o id

enti

fy w

hy d

id t

he a

utho

r m

ake

a ne

w p

arag

raph

… w

hat

is

the

reas

on f

or t

he c

hang

e …

wha

t ch

ange

d? T

C h

as t

he

teac

hers

rea

d si

lent

ly, a

ppro

xim

atel

y th

e fi

rst

five

par

agra

phs

and

asks

the

m t

o id

enti

fy W

HY

the

par

agra

ph c

hang

ed.

Var

ious

rea

sons

em

erge

: cha

nge

in lo

cati

ons,

cha

nge

in t

ime,

ch

ange

in s

peak

er. T

C m

akes

tha

t po

int

that

we

need

to

inst

ruct

exp

licit

ly k

ids

abou

t pa

ragr

aph

chan

ges

and

reas

ons

why

par

agra

phs

chan

ge, v

ia e

xam

ples

. She

als

o m

akes

the

po

int

that

we

need

to

be c

aref

ul a

bout

wha

t st

orie

s/m

ater

ials

w

e se

lect

to

use

for

this

inst

ruct

ion

beca

use

of p

oor

proo

fing

an

d in

corr

ect,

om

itte

d pa

ragr

aphi

ng in

som

e bo

oks.

L t

each

er w

rite

s a

post

-it

note

to

self

. [I

thin

k th

at t

his

note

, al

ong

wit

h on

e or

tw

o ot

hers

, is

his

way

of

reco

rdin

g

D o

ff-t

ask

diad

dis

cuss

ion

wit

h F

nex

t to

her

re:

I n

eed

to g

o to

the

off

ice.

R

espo

nse:

for

wha

t? T

o dr

op o

ff m

oney

… w

hile

M a

nd c

oach

dis

cuss

issu

e M

ra

ised

. Tha

t is

, M is

get

ting

at

wha

t dr

ives

inst

ruct

ion.

Coa

ch s

ays

trus

t th

e fr

amew

ork

(sta

ndar

ds).

Coa

ch r

eite

rate

s th

at t

he D

istr

ict

is d

evel

opin

g a

paci

ng/

inst

ruct

iona

l pla

n fo

r m

ath

for

the

com

ing

year

, thu

s, n

ext y

ear

will

be

easi

er. O

C:

I ob

serv

e th

at D

is s

itti

ng b

ack,

look

s at

wat

ch. C

oach

: So

does

eve

rybo

dy k

ind

of

know

wha

t th

ey n

eed

to w

ork

on …

Res

pons

es f

rom

tea

cher

s ar

e on

the

min

imal

si

de, s

ome

yes/

nods

. Coa

ch a

sks:

So

wha

t do

you

… (

i.e.,

the

coac

h at

tem

pt to

get

te

ache

rs, p

arti

cula

rly

D t

o vo

calis

e so

me

of t

he s

peci

fic

stan

dard

sub

-are

as/s

kills

th

at t

hey

high

light

ed, w

hich

the

y w

ill p

rior

itiz

e/in

stru

ct o

ver

the

next

few

wee

ks.

[OC

: The

coa

ch p

ress

es f

or m

ore

spec

ific

ity

in t

his

mee

ting

tha

n in

pri

or u

pper

G

L m

eeti

ng. G

ood

mov

e, b

ut …

] D

: Jus

t re

view

eve

ryth

ing

… I

kno

w t

hey’

ll ge

t it

[ag

ain,

a n

on-s

peci

fic

resp

onse

]. N

say

s th

at it

’s c

omfo

rtin

g to

kno

w t

hat

the

fram

ewor

k/pa

cing

pla

n is

bei

ng d

evel

oped

in t

ime

for

next

yea

r; t

hat

this

yea

r is

a

loss

and

I h

ave

good

, sm

art

kids

. It’

s re

ally

har

d be

caus

e of

the

spa

n …

N a

nd a

co

uple

of

othe

rs t

hey

shou

ld k

now

mul

tipl

icat

ion

tabl

es b

y G

rade

3. D

con

curs

an

d sa

ys if

the

y do

n’t

they

sho

uld

be k

icke

d ou

t of

sch

ool.

D s

tand

s, s

ayin

g “I

’ve

got

to s

tand

…”

Coa

ch a

nd o

ne t

each

er t

alk

abou

t an

acr

onym

. D t

o co

ach,

I’v

e go

t to

go

drop

off

$ t

o of

fice

, lea

ves.

(K

am,

grad

e-le

vel t

eam

mee

ting)

pers

onal

ly u

sefu

l inf

orm

atio

n fo

r hi

mse

lf, s

uch

as t

he

info

rmat

ion

that

TC

just

des

crib

ed a

bout

tea

chin

g pa

ragr

aphi

ng, r

easo

ns f

or p

arag

raph

cha

nges

, rel

ated

to

the

wri

ting

ski

ll of

inde

ntin

g] …

TC

sta

tes

[in

a su

mm

ing/

conf

irm

ator

y m

anne

r]: “

So

I w

ill m

ake

copi

es o

f …

the

run

-on

sen

tenc

es u

nit

for

you

[oth

er t

each

ers]

and

you

guy

s m

ake

…”

(Elm

, gr

ade-

leve

l tea

m m

eetin

g)


able

5.

(Con

tinue

d)

GR

sch

ools

Com

pari

son

scho

ols

AL

T T

red

irec

ts th

e gr

oup

back

to ta

sk o

f try

ing

to d

eter

min

e w

hat

port

ions

of c

hapt

ers

to k

eep/

drop

… P

rinc

ipal

chi

mes

in

(in

rega

rd t

o po

ssib

le w

ays

to t

each

mat

h sk

ills,

par

ticu

larl

y ot

her

than

form

al s

ched

uled

mat

h ti

me)

: “ …

if y

ou c

an [

take

ti

me

to t

each

a p

arti

cula

r m

ath

skill

] do

at

8:20

[A

.M.]

, but

ev

eryo

ne h

as t

o do

it”.

MT

ref

ocus

es t

he g

roup

by

sum

mar

isin

g: S

o, h

ave

we

deci

ded

that

we

will

do

… A

LT

T

begi

ns t

o m

odel

a t

each

ing

acti

vity

/pro

cedu

re s

he u

ses

wit

h he

r st

uden

ts (

stan

ding

at

her

mat

h bu

lleti

n bo

ard,

coi

ns

acti

vity

). P

rinc

ipal

chi

mes

in: l

ast

wee

k w

hen

I w

as in

___

_’s

clas

s, s

he h

ad k

ids

mak

e up

dif

fere

nt w

ays

of m

akin

g 26

…

AL

T T

[fo

cusi

ng g

roup

by

conc

ludi

ng]:

“…

so

can

we

say

we’

re g

oing

to

do t

ime

and

mon

ey …

one

wee

k …

tha

t le

aves

__

_ w

eeks

for

rem

aini

ng …

(O

ak,

grad

e-le

vel t

eam

mee

ting)

F s

ays

that

nex

t w

eek’

s co

llabo

rati

ve p

lann

ing

mee

ting

will

hav

e so

met

hing

gu

ided

, but

tha

t th

is w

eek

we’

re “

mos

tly

on o

ur o

wn”

. F m

enti

ons

curr

ent/

upco

min

g ta

sk t

hat

teac

hers

nee

d to

do,

indi

cati

ng t

hat

this

tim

e ca

n be

use

d fo

r th

ese

task

s …

Ope

n H

ouse

, Rep

ort

Car

ds, [

Impl

emen

tati

on?

and/

or E

LD

?],

cum

ulat

ive

fold

ers.

I a

sk, s

o yo

u ha

ve r

epor

t ca

rds

to d

o? F

: Not

us

(not

F a

nd M

w

ho a

re o

n sa

me

trac

k), b

ut t

each

ers

on o

ther

tra

ck. M

and

F lo

okin

g at

stu

dent

pa

inti

ngs

… d

iscu

ss/ r

ecom

men

d us

e of

tem

pura

pai

nt v

ersu

s/or

wat

erco

lour

s.

The

tw

o te

ache

rs a

re w

orki

ng in

depe

nden

tly

on s

tude

nt c

umul

ativ

e fo

lder

s,

wri

ting

info

rmat

ion

and

chec

king

info

rmat

ion

in c

umul

ativ

e fo

lder

s …

the

y ch

at

abou

t M

’s w

ife

and

kids

… b

abys

itti

ng a

rran

gem

ents

. (H

all,

grad

e-le

vel t

eam

m

eetin

g)

Lea

der

dist

ribu

tes

“Wri

ting

Scr

ipt”

tha

t gr

ade

leve

l use

s. W

e al

l use

the

“B

erns

tein

Bea

rs”.

She

ask

s if

it’s

OK

if w

e us

e it

ag

ain

… it

’s w

hat w

e’ve

use

d in

the

past

. Lea

der

revi

ews

step

s th

at t

each

ers

will

be

doin

g le

adin

g up

to

asse

ssm

ent

(lea

der

and

othe

rs a

re lo

okin

g at

“W

riti

ng S

crip

t” w

hich

spe

cifi

es

thes

e st

eps)

. In

disc

ussi

ng h

ow t

o ad

min

iste

r th

e as

sess

men

t,

grou

p cl

arif

ies

wha

t to

do/

wha

t no

t to

do

and

gets

to

idea

tha

t th

is is

an

init

ial a

sses

smen

t. L

eade

r di

stri

bute

s “W

riti

ng

Rub

ric”

… E

des

crib

es a

n ex

ampl

e fr

om a

noth

er y

ear/

te

ache

r. O

C: E

see

ms

to p

rovi

de v

ery

solid

exa

mpl

es.

I no

tice

tha

t du

ring

the

init

ial p

orti

on o

f th

e m

eeti

ng t

hat

Mrs

. Z is

pre

pari

ng

som

e ty

pe o

f m

ater

ials

/cut

ting

con

stru

ctio

n pa

per

[i.e

., u

nrel

ated

to

mee

ting

to

pic/

inte

nded

tas

ks].

(K

am,

grad

e-le

vel t

eam

mee

ting)


Tab

le 5

.(C

ontin

ued)

GR

sch

ools

Com

pari

son

scho

ols

Dis

cuss

ion

of w

hat

“det

ail”

mea

ns in

thi

s gr

ade

leve

l’s

wri

ting

. Lea

der

revi

ewin

g “W

riti

ng R

ubri

c”. L

eade

r di

scus

ses

upco

min

g m

eeti

ng a

nd t

hat,

am

ong

othe

r ac

tivi

ties

, we’

ll sc

ore

wri

ting

ass

essm

ents

tha

t da

y. D

iscu

ssio

n of

sch

edul

ing

item

s. 2

:10

Pri

ncip

al d

epar

ts. …

Lea

der

shif

ts to

OC

R, w

hich

is

age

nda

item

2. A

ny c

once

rns?

Mal

e as

ks f

or c

lari

fica

tion

ab

out

som

e w

riti

ng a

sses

smen

t pr

oced

ures

tim

e =

30

min

utes

; will

mod

el s

till

be u

p =

no;

but

web

can

sta

y up

. E

show

s m

ater

ials

tha

t sh

e us

es w

ith

her

stud

ents

; oth

er

teac

hers

vie

w; f

avou

rabl

e re

acti

ons.

2:1

7 M

eeti

ng e

nds.

(P

ine,

gra

de-l

evel

team

mee

ting)


able

5.

(Con

tinue

d)

GR

sch

ools

Com

pari

son

scho

ols

N (

AL

T r

ep f

or t

his

grad

e le

vel)

rev

iew

s la

st m

eeti

ng:

NO

TE

: Ini

tial

ly, I

don

’t s

ee o

r he

ar m

uch

acti

ve r

espo

nses

fro

m t

each

ers;

the

y’re

m

ostl

y ve

ry q

uiet

. The

faci

litat

or d

oes

utili

se th

e ap

proa

ch/a

ppea

l in

atte

mpt

ing

to

get

to g

et t

each

ers

to p

rovi

de in

put/

deci

sion

s, t

hat

she

does

n’t

know

the

ir

stud

ents

, you

kno

w y

our

stud

ents

and

wha

t th

ey s

houl

d/sh

ould

not

be

culle

d. F

or

the

firs

t fe

w w

eek-

by-w

eek

atte

mpt

s to

get

tea

cher

inpu

t, I

alm

ost

have

tha

t “p

ullin

g te

eth”

fee

ling

and

I w

onde

r if

par

t of

the

issu

e is

whe

ther

som

e of

the

te

ache

rs a

re im

plem

enti

ng a

ny o

f Mov

e It

Mat

h, o

r do

ing

so o

n a

limit

ed b

asis

. At

one

poin

t in

the

mee

ting

/bef

ore

or a

fter

I h

ear

gree

n sw

eate

r te

ache

r sa

y in

re

spon

se t

o ei

ther

O’s

(di

stri

ct/ n

atio

nal c

oach

) or

A’s

(sc

hool

-bas

ed c

oach

) qu

ery,

“A

re y

ou o

n a

set

wee

k?”

that

“ye

s” b

ut s

ubse

quen

tly

“We

wer

en’t

rea

lly

doin

g as

muc

h m

ath

as w

e’re

sup

pose

d to

… (

due

to, f

or e

xam

ple,

not

hav

ing

trai

ning

). O

late

r te

lls m

e w

hen

I am

alo

ne w

ith

her

that

the

y ha

d pu

rpos

eful

ly

done

thr

ee t

rain

ings

in t

he s

umm

er, o

ne p

er m

onth

, so

that

the

y co

uld

hit

all

trac

ks’ t

each

ers.

I a

lso

saw

a f

ax d

ated

11-

08 f

rom

O t

o L

ot/A

tha

t lis

ted

item

s ne

eded

for

thi

s m

eeti

ng, i

nclu

ding

list

/num

ber

of t

each

ers

who

had

not

gon

e to

/re

ceiv

ed t

he M

ove

It M

ath

trai

ning

. 11:

08 O

ne s

tude

nt e

nter

s ro

om, h

ands

pap

er

to o

ne t

each

er. T

he o

ne t

each

er d

epar

ts r

oom

for

abo

ut 3

0 se

cond

s. A

bout

one

m

inut

e la

ter,

ano

ther

tea

cher

dep

arts

roo

m f

or a

bout

15

seco

nds.

30

seco

nds

late

r, a

stu

dent

ope

ns d

oor

and

calls

one

tea

cher

’s n

ame.

The

tea

cher

say

s, “

Go

find

Mrs

. Gom

ez”

and

repe

ats

that

phr

ase.

11:

12 O

: “D

o yo

u fe

el c

omfo

rtab

le

…?”

(sk

ippi

ng)

C: R

egar

ding

Mon

ster

add

itio

n: “

You

’re

not

aski

ng t

hem

to

mas

ter

it. Y

ou’r

e ju

st in

trod

ucin

g it

.” A

tea

cher

had

com

men

ted

that

thi

s w

as a

to

uchy

one

, tha

t M

onst

er m

ath

is d

iffi

cult

for

the

kid

s. B

tea

cher

ask

s: “

How

im

port

ant

is S

cott

-For

esm

an?”

B t

each

er m

akes

poi

nt t

hat

afte

r do

ing

man

ipul

ativ

es a

nd …

tha

t ki

ds a

re t

ired

of

it a

nd t

here

fore

is it

nec

essa

ry t

o do

w

hole

Mov

e it

Mat

h pa

ge/a

ll pr

oble

ms.

R&

B t

each

er: “

In y

our

scho

ol, y

ou w

ere

man

agin

g tw

o m

ath

prob

lem

s as

wel

l?”

“Bec

ause

I a

m w

onde

ring

…”

O: N

o, it

ju

st h

appe

ned.


able

5.

(Con

tinue

d)

GR

sch

ools

Com

pari

son

scho

ols

•D

aily

ora

l exe

rcis

es …

Coa

ch w

ill p

rovi

de•

Tw

o te

ache

rs s

hare

d st

rate

gies

for

seq

uenc

ing

Spa

rks

disc

ussi

on, L

B t

each

er s

ays

… “

feel

fru

stra

ted”

•Id

enti

fied

new

nee

d …

new

obj

ecti

ve …

•N

shi

fts

to w

hat

this

mee

ting

will

add

ress

. He

plac

es O

H•

Nee

d =

incr

ease

stu

dent

kno

wle

dge

of s

eque

ncin

g w

hen

wri

ting

sto

ry

O: A

s P

roje

ct G

rad

scho

ol, y

ou’r

e su

ppos

ed t

o fo

llow

Mov

e it

Mat

h an

d us

e S

cott

-For

esm

an a

s su

pple

men

t. W

e do

n’t

real

ly c

are

if y

ou u

se S

cott

-For

esm

an

or …

or …

as

long

as

you’

re a

ddre

ssin

g th

e M

ove

it M

ath

focu

s (i

.e.,

hav

ing

all

your

com

pone

nts,

par

ts o

f le

sson

incl

udin

g w

arm

-up,

etc

.) (

Lot

, gr

ade-

leve

l tea

m

mee

ting)

•O

bjec

tive

= B

y en

d of

sch

ool y

ear

75%

of s

tude

nts

WB

AT

to

wri

te 4

or

mor

e se

nten

ces

to d

escr

ibe

begi

nnin

g,

mid

dle,

end

.N

lead

s te

ache

rs t

o to

day’

s ta

sk o

f an

alys

ing

one

low

, one

m

ediu

m, o

ne h

igh

stud

ent

wor

k sa

mpl

es t

o de

term

ine

stre

ngth

s an

d w

eakn

esse

s. O

C: I

t ap

pear

s th

at e

ach

teac

her

has

sele

cted

one

hig

h, o

ne lo

w, a

nd o

ne m

iddl

e st

uden

t su

mm

ary

from

the

ir o

wn

clas

s. G

ood

sign

in t

erm

s of

peo

ple

actu

ally

bro

ught

the

sam

ples

as

agre

ed u

pon/

dire

cted

in p

rior

m

eeti

ng/c

omm

unic

atio

ns.

N s

how

s on

OH

a “

1” e

xam

ple

of s

tude

nt w

ritt

en s

umm

ary

of t

he T

hree

Litt

le P

igs.

Sho

ws

a “2

” ex

ampl

e. N

sho

ws

a hi

gh

(“3”

) R

ask

s T

s w

hat

are

som

e of

the

str

engt

hs t

hey

see

in t

he

sam

ples

. Ts

iden

tify

abo

ut 7

dif

fere

nt s

peci

fic

stre

ngth

s w

ith

mul

tipl

e te

ache

rs r

espo

ndin

g, o

ne a

t a

tim

e …

OC

: Mor

e go

od s

igns

= m

ulti

ple

teac

hers

par

tici

pati

ng a

nd t

each

ers

utili

se c

omm

on/e

ffic

ient

cou

rtes

ies

of s

peak

ing

one

pers

on a

t a

tim

e. S

ome

of t

he s

tren

gths

tha

t te

ache

rs v

ocal

ise–

and

N

wri

tes

thes

e on

the

OH

… (

Elm

, gr

ade-

leve

l tea

m m

eetin

g)


of meeting topics; (k) scheduled, weekly, “hands-off” (i.e., protected fromcompeting demands and conducted as scheduled) versus “more loosely” scheduledmeetings that were frequently cancelled, curtailed, rescheduled, changed at the lastminute, or otherwise disrupted.

Staff at GR and comparison schools reported that the Content Standards forCalifornia Public Schools (California State Department of Education, n.d.) consti-tuted their school-wide academic goals. They also stated that improving students’scores on annual standardised tests was a school-wide goal. However, only GLTs atGR schools actually formulated, wrote, disseminated, and evaluated “more specific”academic goals based on the Standards. Some of these GLTs also focused theirinstruction on these specific goals, in a “planful” manner, by utilising a systematicprocess they acquired via GR training (see Table 6). GLTs at comparison schoolsdid not have or execute such systematic processes during their meetings.

Finding 2. The GR Model established tighter linkages between teachers andadministrators in their efforts to focus on academic goals and improve students’academic achievement. First, teachers at GR schools were much more visible andregular participants in their schools’ academic leadership teams (termed ALTs atGR schools). Each of the four GR schools included at least one teacher from eachgrade level on their school-wide ALT. Of the three comparison schools, Hall hadzero teachers on its leadership team, Lot did not have a formal leadership team, andKam sometimes included one or two teachers on its team on an as-needed basis.Second, principals at GR schools attended and participated more consistently atGLT meetings and teachers’ professional development sessions than principals atcomparison schools. Principals at most GR schools demonstrated greater awareness,focus, and participation in the day-to-day academic plans and actions of teachers ateach grade level. The tighter academic linkages between teachers and administratorsat GR schools facilitated more effective execution of goal-directed plans than atcomparison schools, where the evaluator observed more frequent “slippage”between intended actions and actual implementation of academic initiatives.

RQ 3: What does implementation of the GR Model do to impact academic achievement?

Finding 1. GR Model implementation impacted students’ academic achievementby developing settings and processes whereby educators’ behaviours and instruc-tional processes became more focused and produced visible improvements instudents’ academic achievement and attainment of academic goals. teachers’ attri-butions for academic gains, teachers’ attitudes toward purposeful instructional tasks,teachers’ instructional efficacy, and teachers’ expectations for themselves and forstudents’ academic achievement changed when, and to the extent that, teachersexperienced—frequently, punctually, and directly—visible improvements inacademic achievement associated with their “results-producing” behaviours andinstructional processes.


able

6.

GR

em

phas

is o

n sy

stem

atic

pla

nnin

g an

d in

stru

ctio

n (F

ocus

ing

on a

nd A

ddre

ssin

g C

omm

on S

tude

nt N

eeds

in G

rade

Lev

el M

eeti

ngs:

T

he 7

Im

port

ant T

asks

/Ste

ps …

in a

Nut

shel

l)

Tas

kG

oal

Thi

ngs

to r

emem

ber

1Id

enti

fy a

nd c

lari

fy s

peci

fic

and

com

mon

stu

dent

ne

eds

to w

ork

on t

oget

her

●O

ptio

ns f

or id

enti

fyin

g ne

eds:

Sta

ndar

ds, S

tanf

ord

9 co

nten

t cl

uste

rs,

Beg

inni

ng o

r M

idye

ar a

sses

smen

t re

sult

s, C

urri

culu

m (

read

ing

or

mat

hem

atic

s se

ries

)●

If y

ou h

ave

BM

Es,

use

tho

se t

o id

enti

fy n

eeds

●C

onne

ct/c

lari

fy n

eeds

usi

ng s

tand

ards

●C

lari

fy n

eeds

by

anal

ysin

g st

uden

t w

ork

●W

rite

a b

rief

des

crip

tion

of

the

need

2F

orm

ulat

e a

clea

r ob

ject

ive

for

each

com

mon

nee

d an

d id

enti

fy r

elat

ed s

tude

nt w

ork

to b

e an

alys

ed●

Rev

iew

the

com

pone

nts

of c

lear

obj

ecti

ves:

Wha

t do

you

wan

t st

uden

ts t

o do

, und

er w

hat

circ

umst

ance

s, a

nd w

ith

wha

t, if

any

, kin

d of

sup

port

or

set

up?

How

man

y st

uden

ts d

o yo

u ex

pect

to

mee

t th

e ob

ject

ive

by w

hen,

and

w

hat

do y

ou w

ant

to s

ee t

o ev

iden

ce s

ucce

ss o

r m

aste

ry (

desi

red

qual

itie

s)?

●F

ollo

w a

n ex

ampl

e w

hen

wri

ting

an

obje

ctiv

e:S

ampl

e cl

arifi

ed n

eed:

For

Gra

de 1

, wri

ting

flu

ency

is b

eing

abl

e to

thi

nk

abou

t a

topi

c, f

ind

thin

gs t

o sa

y ab

out

the

topi

c, a

nd e

ffec

tive

ly u

se le

tter

s to

wri

te w

hat

you

wan

t to

say

, suc

h th

at s

omeo

ne e

lse

can

read

and

un

ders

tand

it.

Sam

ple

obje

ctiv

e: G

iven

fam

iliar

topi

cs, b

y th

e en

d of

Jan

uary

, mos

t Gra

de 1

st

uden

ts (

say,

75%

) w

ill b

e ab

le t

o co

nsis

tent

ly p

rodu

ce jo

urna

l ent

ries

du

ring

dai

ly jo

urna

ling

tim

e of

fou

r or

mor

e re

late

d se

nten

ces

that

sta

y on

to

pic,

sho

w r

easo

nabl

e th

ough

t, a

nd a

re r

eada

ble.


able

6.

(Con

tinue

d)

Tas

kG

oal

Thi

ngs

to r

emem

ber

3Id

enti

fy a

nd a

dopt

a p

rom

isin

g in

stru

ctio

nal f

ocus

to

add

ress

eac

h co

mm

on n

eed

•C

onsi

der

firs

t w

hat

you

alre

ady

have

•C

onsi

der

som

ethi

ng a

bout

whi

ch s

omeo

ne in

the

gro

up h

as e

xper

tise

•C

onsi

der

som

ethi

ng a

bout

whi

ch y

ou c

an g

et p

ublis

hed

info

rmat

ion

•C

onsi

der

som

ethi

ng a

bout

whi

ch y

ou c

an s

ecur

e ou

tsid

e as

sist

ance

•U

se t

hese

que

stio

ns t

o ga

uge

the

prom

ise

of t

he in

stru

ctio

nal f

ocus

:-

Can

you

gen

eral

ly e

xpla

in w

hat

it is

, and

how

you

do

it in

the

cl

assr

oom

?-

Can

you

fit

it in

to y

our

exis

ting

pro

gram

me?

-C

an y

ou g

ener

ally

exp

lain

why

it w

ill li

kely

hel

p ad

dres

s th

e ob

ject

ive

(ass

umin

g yo

ur t

eam

wor

ks h

ard

to im

plem

ent

it w

ell)

?-

Is it

som

ethi

ng t

hat

a re

ason

able

num

ber

of f

olks

on

your

tea

m w

ill t

ry?

-C

an y

ou s

ecur

e th

e re

sour

ces

nece

ssar

y to

impl

emen

t it

as

a te

am?

•U

se w

orks

heet

to

reco

rd t

asks

1, 2

, 3 (

page

39

of G

R M

anua

l 200

1)4

Pla

n an

d co

mpl

ete

nece

ssar

y pr

epar

atio

n to

try

the

in

stru

ctio

nal f

ocus

in t

he c

lass

room

•P

lann

ing

and

prep

arat

ion

are

ongo

ing

and

cycl

ical

•In

vent

ory

the

nece

ssar

y pl

anni

ng a

nd p

repa

rati

on a

nd s

ecur

e co

mm

itm

ents

•G

et c

oncr

ete—

deve

lop

actu

al w

ritt

en p

lans

and

nee

ded

mat

eria

ls•

Get

con

cret

e—do

moc

k or

rea

l dem

onst

rati

ons

so p

eopl

e ca

n ob

serv

e an

d di

scus

s T

HE

TE

AC

HIN

G


able

6.

(Con

tinue

d 2)

Tas

kG

oal

Thi

ngs

to r

emem

ber

5D

eliv

er I

nstr

ucti

on: M

ake

cons

iste

nt a

nd g

enui

ne

effo

rts

to t

ry t

he t

eam

’s in

stru

ctio

nal f

ocus

in t

he

clas

sroo

m

•T

he c

halle

nge:

thi

s st

ep is

not

tak

en d

urin

g a

mee

ting

but

in e

ach

teac

her’

s cl

ass

•T

ry t

o ge

t a

publ

ic c

omm

itm

ent

duri

ng m

eeti

ngs

from

eac

h te

am m

embe

r to

try

the

inst

ruct

iona

l foc

us in

the

cla

ssro

om•

Iden

tify

indi

vidu

als

who

will

bri

ng s

tude

nt w

ork

back

to

the

subs

eque

nt

mee

ting

or

will

be

prep

ared

to

repo

rt o

n th

eir

clas

sroo

m e

ffor

ts•

Rem

ind,

rem

ind,

rem

ind

(i.e

., “

Don

’t f

orge

t w

e al

l agr

eed

this

wee

k to

try

…

”)6

Ana

lyse

stu

dent

wor

k to

: (a)

see

whe

ther

the

ob

ject

ive

is b

eing

met

, (b)

bet

ter

unde

rsta

nd t

he

need

, and

(c)

eva

luat

e in

stru

ctio

n

•W

ith

rega

rd t

o ge

ttin

g pe

ople

to

brin

g st

uden

t w

ork:

spe

cify

ahe

ad o

f ti

me

wha

t sp

ecif

ic w

ork

to b

ring

, fro

m w

hich

tea

cher

s an

d fo

r w

hich

stu

dent

s•

Whe

n an

alys

ing

stud

ent

wor

k: u

se o

verh

eads

, foc

us o

n th

e ob

ject

ive,

and

id

enti

fy s

tren

gths

and

wea

knes

ses

in t

he w

ork;

the

n co

nnec

t th

e st

reng

ths

to w

hat

wor

ked

in t

he in

stru

ctio

n an

d th

e w

eakn

esse

s to

wha

t ne

eds

to b

e ad

dres

sed

in s

ubse

quen

t le

sson

s•

Use

a c

hart

like

the

one

on

page

37

of G

R M

anua

l, 20

01 t

o gu

ide

the

anal

ysis

7R

eass

ess:

con

tinu

e an

d re

peat

cyc

le o

r m

ove

on t

o an

othe

r ar

ea o

f ne

ed?

•C

entr

al q

uest

ion:

Is

the

obje

ctiv

e be

ing

met

?•

Als

o: A

re t

eam

mem

bers

will

ing

to s

usta

in e

ffor

ts o

n th

is n

eed

and

obje

ctiv

e?•

Gui

delin

e: I

f a

maj

orit

y of

the

mem

bers

wan

t to

sta

y th

e co

urse

, the

n st

ay

the

cour

se


Finding 2. GR implementation shifted some educators’ attributions for studentachievement toward specific, teacher-implemented, instructional actions and plan-ning processes, and away from teacher and student traits, and non-instructionalexplanations (see Table 7). Teachers in schools and GLTs where GR implementa-tion was strong frequently attributed student achievement, or lack thereof, tospecific instructional actions or processes they had or had not instituted. Teachers inschools and GLTs where GR implementation was weak were more likely to attributestudent achievement to global factors or student traits, such as experience andknowledge, socio-economic conditions, inexperience with the English language,academic inability, lack of readiness, and inadequate parental involvement.

For some educators, GR’s emphasis on systematic instruction cycles (Table 6)altered their instructional habits and beliefs about what constituted good teaching.For other educators, GR implementation affirmed these habits and beliefs. In bothinstances, educators clarified and raised their expectations for performing essentialtasks needed to promote student achievement. In educational contexts fraught withcompeting and shifting demands, GR implementation helped many educators,individually and collectively, to prioritise goals and focus instructional effortsaccordingly. Some teachers shifted from strict adherence to timeline-driven or page-sequenced coverage of materials based on actual mandates or perceived pressures.They recognised increasingly the implications of purposeful planning and “teachingless, better” instead of “covering” material. Similarly, GR implementation crystal-lised, for most principals, the importance of focusing on academic goals and essen-tial tasks that most directly impact student achievement. GR “reminded” principalsnot to get distracted by the “operations side” of their job, or by the gauntlet ofnumerous, emerging, competing demands.

Finding 3. GR implementation fostered a group ethos among some teams of grade-level teachers and ALT representatives—a collective willingness and commitment toformulate, adapt, implement, and evaluate instructional processes that targetedstudent achievement. This group ethos was revealed during many, but not all, GLTand ALT meetings via participants’ focused academic discourse, systematic plan-ning, purposeful use of assessment data, and agreements to implement and evaluategoal-directed teaching strategies. By providing adequate time and assistance (e.g.,substitute teachers and class coverage) for teachers to analyse student work and eval-uate assessment data with their peers, GR implementation altered teachers’ attitudestoward such tasks. Teachers increased and improved their analysis of assessmentdata to better inform important instructional decisions that impacted studentachievement. Viewing and interpreting information—especially student work prod-ucts and test results that provided teachers with timely evidence of academicimprovements—increased teachers’ instructional efficacy and promoted attributionsthat their instructional decisions and actions improved student achievement. Dataindicated that GR Model implementation: (a) increased teacher’s willingness toshare assessment data with colleagues; (b) altered teachers’ understanding and


able

7.

Com

pari

son

of t

each

ers’

att

ribu

tion

s fo

r st

uden

t ac

hiev

emen

t

Str

onge

r G

R M

odel

impl

emen

tati

on a

ssoc

iate

d w

ith

teac

her

belie

fs t

hat

inst

ruct

ion

acco

unte

d fo

r st

uden

t ac

hiev

emen

tW

eake

r/no

n-im

plem

enta

tion

of

GR

Mod

el a

ssoc

iate

d w

ith

teac

her

belie

fs t

hat

fact

ors

othe

r th

an in

stru

ctio

n ac

coun

ted

for

stud

ent

achi

evem

ent

A …

mos

t of

the

gra

de le

vels

wer

e do

ing

dem

os [

righ

t]. T

each

ers

wou

ld

volu

ntee

r if

som

ethi

ng w

as w

orki

ng w

ell t

o de

mo

it. S

o, p

art

of t

hese

gra

de

leve

l mee

ting

s st

arte

d be

com

ing

dem

onst

rati

on t

ime-

doin

g lit

tle

min

i-le

sson

s…W

e sh

ared

dif

fere

nt le

sson

s …

som

eone

wou

ld d

emon

stra

te a

le

sson

–wha

teve

r’s

wor

king

in t

he c

lass

room

. So,

I t

hink

it w

as a

lso

very

he

lpfu

l for

us

… a

nd t

hen,

whe

n yo

u fi

nd o

ut w

hat’

s w

orki

ng in

one

cla

ss,

then

you

can

try

it in

the

oth

er c

lass

room

, and

so

even

tual

ly e

very

body

’s

doin

g th

e sa

me

thin

g be

caus

e it

wor

ks.

Com

men

t abo

ut a

cade

mic

ach

ieve

men

t and

we

know

abo

ut th

e st

uden

ts’ a

cade

mic

ach

ieve

men

t (i

.e.,

how

stu

dent

s ar

e do

ing)

…

sta

tem

ents

abo

ut lo

ss o

f con

trol

, “w

e’re

pro

fess

iona

ls”,

don

’t

need

to

have

peo

ple

telli

ng u

s w

hat

to d

o, w

e kn

ow, w

e ha

ve

degr

ees,

mas

ter’

s de

gree

s …

(F

ield

not

es,

Fir

, gr

ade-

leve

l tea

m

mee

ting)

A …

it g

oes

back

to

som

ethi

ng t

hat

I sa

id e

arlie

r. U

m, y

eah,

wit

h w

hate

ver

pres

sure

and

wha

teve

r do

wns

ide,

we

are

show

ing

resu

lts.

(T

rans

crip

t, P

ine,

te

ache

r fo

cus

grou

p)

[Tea

cher

] …

Say

s th

ese

kids

are

n’t

read

y fo

r w

hat

we’

re a

skin

g th

em t

o do

… A

nd I

hav

e ei

ght

stud

ents

who

don

’t s

peak

E

nglis

h, d

on’t

kno

w a

nyth

ing

abou

t w

riti

ng …

the

y do

n’t

get

it.

[Pri

ncip

al a

sks

teac

her]

In

gene

ral o

r ju

st w

riti

ng?

Tea

cher

: In

gene

ral.

(Fie

ld n

otes

, F

ir,

grad

e-le

vel t

eam

mee

ting)

2:23

. T h

olds

up

som

e of

the

bar

cha

rt r

esul

ts s

choo

l wid

e. H

e sa

ys t

hat

he

did

not

mak

e co

pies

of

this

for

gro

up. H

e m

enti

ons

that

(st

uden

ts’

perf

orm

ance

on)

hig

h fr

eque

ncy

wor

ds a

ctua

lly d

ropp

ed fr

om b

egin

ning

to

mid

-yea

r as

sess

men

t, e

spec

ially

in G

rade

s 4

and

5.

Coa

ch d

istr

ibut

es f

irst

one

-pag

e do

cum

ent

wit

h st

anda

rdis

ed

test

res

ults

fro

m la

st y

ear.

Tea

cher

s no

te t

hat

3rd

Gra

de s

core

s fr

om la

st y

ear

wer

e hi

gh a

nd r

elat

e th

at t

his

year

’s g

roup

(i.e

.,

who

are

now

the

ir G

rade

4 s

tude

nts)

sho

uld

do w

ell o

n te

st t

his

year

. Mal

e T

sta

tes

reas

on w

hy t

he s

core

s w

ere

good

/his

st

uden

ts a

re d

oing

wel

l in

mat

h is

bec

ause

mos

t of

the

/his

st

uden

ts a

re m

ale.

He

also

sta

tes

that

tha

t’s

why

the

rea

ding

sc

ores

are

low

; gir

ls d

o be

tter

wit

h re

adin

g, m

ales

do

bett

er w

ith

mat

h …

(F

ield

not

es,

Kam

, gr

ade-

leve

l tea

m m

eetin

g)


able

7.

(Con

tinue

d)

Str

onge

r G

R M

odel

impl

emen

tati

on a

ssoc

iate

d w

ith

teac

her

belie

fs t

hat

inst

ruct

ion

acco

unte

d fo

r st

uden

t ac

hiev

emen

tW

eake

r/no

n-im

plem

enta

tion

of

GR

Mod

el a

ssoc

iate

d w

ith

teac

her

belie

fs t

hat

fact

ors

othe

r th

an in

stru

ctio

n ac

coun

ted

for

stud

ent

achi

evem

ent

R a

sks:

Are

peo

ple

doin

g w

ord

wal

l eve

ryda

y?O

C: R

see

ms

to p

rovi

de o

n-ta

rget

que

stio

ns/ c

omm

ents

tha

t fo

cus

on

anal

ysis

of

the

issu

es. (

Fie

ld n

otes

, O

ak,

grad

e-le

vel t

eam

mee

ting)

T s

ays

nobo

dy is

goi

ng to

han

d yo

u a

blue

prin

t … it

has

to c

ome

from

us

…

one

reas

on w

hy w

e’re

doi

ng s

o m

uch

bett

er t

han

1995

. It’

s a

huge

cha

nge

sinc

e 19

95; i

t’s

not

just

per

sona

litie

s, it

’s h

ow w

e vi

ew t

he s

tude

nts’

wor

k/w

ork.

(F

ield

not

es,

Oak

, ac

adem

ic le

ader

ship

team

mee

ting)

B c

omm

ents

abo

ut w

hen

kids

are

/are

not

rea

dy

“dev

elop

men

tally

” fo

r w

riti

ng t

asks

… r

efer

ence

s to

the

ir (

lack

of

) E

nglis

h re

adin

g/E

nglis

h la

ngua

ge p

rofi

cien

cy …

B m

akes

re

fere

nce

to O

pen

Cou

rt s

core

s …

B s

ays

we

have

a g

oal/f

ocus

on

voc

abul

ary.

The

n di

scus

sion

of

com

preh

ensi

on s

core

s be

ing

not

too

good

. T s

ays

she

thin

ks it

was

the

sto

ry …

(F

ield

not

es,

Fir

, gr

ade-

leve

l tea

m m

eetin

g)


expectations about the purposes of assessment data; (c) fostered an “improvementover time” versus a “one-shot” orientation for collecting, analysing, and using data;and (d) shifted grade-level teams from talking about indicators to systematic analysisand actual use of indicators to plan and implement instruction. The following exam-ple illustrates aspects of this finding. A teacher at Pine explained how GR-initiated,BME writing assessments changed the GLT’s use of assessment tools.

Our grade-level meetings, which were supposed to be geared toward improving scores,really had no relation to what tests we were giving. But now this year, since we’re able tochoose a goal to pick with the writing assessment … Those [assessments] have becomea valuable tool, whereas previously, I was just given them and that was it. They had novalue to me at all. (Teacher focus group)

Finding 4. GR implementation, through collaborative goal setting, analysis of indi-cators, and reflection on teacher-controlled instructional variables, impacted teach-ers’ expectations for student achievement. Some teachers’ raised their expectationsfor student achievement after they viewed student work or analysed data thatprovided explicit evidence of students’ academic improvement. However, mostteachers’ expectations did not appear to rise dramatically. Modest expectations forstudent achievement were reflected in teachers’ initial specification and subsequentrevision of academic goals, objectives, and criteria. Document retrieval indicatedthat GLTs frequently set a criterion of “75% of the students” when formulatingacademic goals. More notably, the practice of collaborative goal setting brought tothe surface teacher expectations for student achievement, and prompted someteachers to examine implications of their individual assumptions and collectiveexpectations, as reflected in the following exchange between teachers at OakElementary:

T1: I give them a piece of writing almost every night, or some kind of poem.T2: And you’re expecting them now to be able to able to look at a rubric and get a

three or a four?T3: I see an improvement, you know, big time … But I think some kids will never

have the ability to be a four in my mind.T1: At the fifth-grade level?T3: Yeah, because, um, you know, whatever factors … there are; their families or, I

have a girl who’s only been here, you know, for a few months … let’s say fromArgentina … you know, that’s just the way it is.

T1: But I think kids that have been here … that have had exposure to this writingprocess, exposure, it’s only going to help them.

T2: … I think the first graders now, let’s say in five years when they come to fifthgrade, hopefully they’ll be, uh, able to achieve better because they would havebeen exposed to the process.

T1: Yeah. You know, I’m a little dismayed to hear teachers say that they can neverachieve that level.

Finding 5. GR implementation required teachers to assume academic leadershiproles and to chart the academic course and outcomes of their schools. As teachers


performed leadership tasks that impacted their colleagues and promoted academicachievement, this altered teachers’ professional responsibility, instructional efficacy,and collegial accountability, particularly among teachers who served as ALT repre-sentatives for their GLTs. This finding is supported strongly via stark contrastsbetween what constituted a GLT meeting before and after GR implementation, andat GR versus comparison schools (see Table 5). Data collection identified manyexamples associated with the transformation of GLT meetings from “chat sessions”to settings where teachers took responsibility to meet collaboratively and focus onimproving academic achievement. Germane examples appeared in ALT meetings,too. These examples reflected teachers’ attitudes toward performing important tasksthat impacted student achievement, including sharing expertise with colleagues,supporting and pressuring colleagues, preparing instruction, utilising meeting time,and completing professional tasks.

A heightened sense of professional responsibility was illustrated vividly in twocases in which staff confronted very challenging issues. In the first case, GR andschool staff formed and implemented a strategy designed to get “reluctant” teach-ers to participate more actively in GLT efforts to improve students’ writing. In thesecond case, GR and school staff formed and implemented an interventiondesigned to assist a “struggling” upper-grade GLT. In both cases, ongoing exter-nal assistance from GR staff was paired with internal expertise and leadership ofschool staff. This melding of assistance and leadership triggered a sense ofurgency, agency, and responsibility among staff (Earl & Lee, 1998). It enabledstaff to confront problematic issues that had inhibited necessary changes ininstruction.

RQ 4: What helps and hinders schools’ implementation of the GR Model?

Five factors facilitated or inhibited GR Model implementation. The first factor thataffected GR Model implementation was combined leadership of school administra-tors and teachers who represented their GLT on school-wide ALTs. Administratorsand ALT representatives at Oak and Pine focused more consistently on academicgoals than their colleagues at Elm and Fir, as evidenced in their respective actions,participation, and discourse during GLT and ALT meetings. Administrators at Oakand Pine attended GLT meetings more frequently than their colleagues at Pine andFir. Administrators at Elm were distracted frequently by the day-to-day operationaldemands of their jobs. Administrators at Fir and Elm more frequently used andresponded to “walkie-talkie” communications during GLT and ALT meetings.These off-topic communications interrupted the flow and academic focus of themeetings. Elm’s Principal reported being “overwhelmed” by the combined academicand operational demands of the job. Some ALT representatives (i.e., teachers whowere GLT leaders) at Elm tended to “pick up” some of the leadership roles andresponsibilities for implementing GR activities. The principal at Fir was not asdistracted as the principal at Elm, but the leadership provided by ALT representa-tives at Fir was the least effective of all GR schools. Consequently, the overall rating


for the leadership element at Fir was 2—slightly lower than Elm’s rating of 3, andlower still than the rating of 3 to 4 for both Oak and Pine.

The second factor that impacted GR implementation was the relative frequencyand continuity of expert assistance provided by GR staff. Direct observations of GRstaff’s assistance confirmed school staffs’ repeated assertions that GR staff membersprovided essential expertise and leadership. GR assistance was strong over time atElm, Pine, and Oak. At Fir, a GR staff member provided frequent assistance to theprincipal and staff during Year 1. However, that assistance was provided much lessfrequently during subsequent years. The principal and some teachers at Fir reportedthat they “missed” this external assistance. Ratings and direct observations of GLTand ALT meetings at Fir confirmed this finding (see Table 4).

A third factor that affected GR implementation was school staffs’ readiness forchange, which varied by individual teachers and GLTs. Most nascent and veteranteachers were optimistic about school change initiatives and programmes. Theyinvested energy to implement GR Model activities and other initiatives. However, afew veteran teachers were pessimistic. They limited their investment and participa-tion based on past experiences with short-lived initiatives and programmes that“come and go”. In addition, most staff at GR schools were relatively unencumberedby “baggage” or past experiences that negatively impacted current relationships andjob performance. However, lingering issues, differing expectations, and idiosyncratichabits among some GLTs and individuals diverted valuable energy and time fromschool change efforts.

A fourth factor that impacted GR implementation was concurrent job demands.As stated previously, K-3 GLTs implemented GR procedures more effectively thantheir colleagues in Grades 4 and 5. Unlike their K-3 colleagues who had imple-mented a new structured reading programme the preceding year, teachers in theupper grades, at three of four GR schools, were implementing those sameprogrammes for the first time. Upper-grade teachers reported consistently that theyexpended extraordinary time implementing this new reading programme. First-yearimplementation demands, combined with concurrent first-year implementation of anew mathematics programme, proliferation of assessments, and difficulties resolvingpacing plans, accounted, in part, for greater stress and less effective GR implementa-tion among upper versus lower GLTs.

A fifth factor that impacted GR implementation was the type of operating sched-ule used at each school. Year-round multi-track scheduling—a response to over-crowding and shortages of school buildings in this dense urban district—clearlycomplicated communication, planning, and scheduling at Elm, Fir, and Oak (andcomparison schools), but not at Pine, which used a traditional, 9-month single-trackschedule. Staff at Elm, Oak, and Fir expended considerable time during GLT meet-ings, ALT meetings, faculty meetings, and professional development sessions tryingto determine how teachers who were “off-track” could be informed, accommodated,or trained on GR and related activities. Teachers and administrators consistentlyexpressed concerns about the “hand-off” and continuity of initiatives betweenincoming and outgoing tracks. Multi-track scheduling made “teacher buy-in” more


challenging because one-third of the staff and their students were not present; theywere on their off-track vacation period. Consequently, only two-thirds of the staffwere present, at any given time, when important decisions were made, such as whenGLTs formulated academic goals and agreed to use corresponding instructionaltechniques.

RQ 5: What has changed as a result of GR Model implementation from the perspectives of participants and the external evaluator?

According to participants and the external evaluator, GR implementation increasededucators’ focus on academic learning and on getting academic results in fourimportant ways (see Table 8). First, GR transformed GLT meetings, from settingswhere non-academic topics and activities consumed time into settings where system-atic academic planning, instructional modeling, analysis of student work, and otherpurposeful, goal-directed tasks informed teachers’ instruction. Second, GR assistedand challenged teachers to provide leadership in charting the academic course of theschool through participation in newly created ALTs. Third, GR staff assisted andchallenged administrators and staff to prioritise and emphasise purposeful, goal-directed efforts aimed at improving teaching and learning. Fourth, GR fosteredsystematic collection and use of assessment data, particularly BME writingassessments, in ways that more punctually and more effectively informed teachers’instructional decisions.

Discussion

In this final section, we discuss limitations of this external evaluation study, as wellas implications of this study’s findings for school reform in general, and morespecifically for understanding what goes on inside the black box of reform.

Limitations of the Evaluation Study

Important limitations of this study relate to the case-study approach used in thisprocess-focused evaluation. Case studies, inherently, must be bounded (Merriam,1998). We purposefully selected seven cases—four GR schools and three demo-graphically similar comparison schools. Thus, our findings, discussion, and implica-tions must be placed into context. One obvious limitation is that our findingsemanate strictly from primary schools. We did not and have not investigated howGR processes might operate in middle schools or high schools. So, we are left towonder if and how GR processes might apply to such schools. Some researchershave reported that educational reform processes, and even evaluating suchprocesses, are more challenging in middle schools and high schools compared withprimary schools (Greene & Lee, 2006).

Another major limitation of this study is that the external evaluator conductedrelatively few observations of teachers instructing students in their classrooms.


able

8.

Par

tici

pant

s’ p

ersp

ecti

ves

on c

hang

es r

esul

ting

fro

m G

R im

plem

enta

tion

Gra

de-

leve

l tea

m m

eeti

ngs

tra

nsf

orm

edA

dm

inis

trat

ors

pri

orit

ised

an

d f

ocu

sed

“We’

re u

sing

our

gra

de-l

evel

mee

ting

s to

foc

us o

n in

stru

ctio

n no

w…

the

grad

e-le

vel m

eeti

ngs

are

very

wel

l pla

nned

…ha

ve

agen

das…

grad

e-le

vel m

eeti

ngs

are

no lo

nger

[us

ed f

or]

plan

ning

tr

eats

and

ass

embl

y pe

rfor

man

ces,

unf

ortu

nate

ly [

som

e-on

e ch

uckl

es].

But

it’s

goo

d …

for

the

stud

ents

bec

ause

it’s

not

a lo

t of

do

wn

tim

e at

the

se m

eeti

ngs…

focu

s on

[st

uden

ts’ a

cade

mic

] ne

eds

… m

ost

of t

he g

rade

leve

ls w

ere

doin

g de

mos

. Tea

cher

s w

ould

vo

lunt

eer

if s

ome-

som

ethi

ng w

as w

orki

ng w

ell t

o de

mo

it. S

o pa

rt o

f th

ese

grad

e-le

vel m

eeti

ngs

star

ted

beco

min

g de

mon

stra

tion

tim

e—do

ing

littl

e le

sson

s.”

(Pin

e fo

cus

grou

p)

[GR

] “h

elpe

d m

e fo

cus”

giv

en a

ll of

the

man

y th

ings

that

dra

w p

rinc

ipal

s aw

ay f

rom

inst

ruct

ion.

She

say

s, “

I di

dn’t

kno

w h

ow t

o le

ad”

in t

he p

ast

whe

n sh

e ca

me

on a

s a

prin

cipa

l, bu

t th

at B

ruce

/[G

R]

show

ed h

er h

ow,

the

spec

ific

s of

lead

ing

peop

le a

nd f

ocus

ing

them

on

inst

ruct

ion.

She

sh

owed

the

AL

T (

how

to le

ad a

nd fo

cus)

, the

n th

ey le

ad a

nd d

id li

kew

ise

wit

h th

eir

colle

ague

s …

whe

n sh

e fi

rst

chan

ged

from

gra

de-l

evel

cha

irs

form

at t

o A

LT

, tha

t te

ache

rs s

till

used

the

mee

ting

s as

com

plai

nt

sess

ions

and

to

deal

wit

h th

e m

any

non-

inst

ruct

iona

l ite

ms

… m

anag

ed

to g

et t

hem

focu

sed

on in

stru

ctio

nal g

oals

… v

ia p

ress

ure.

(P

ine

prin

cipa

l in

terv

iew

)…

hav

ing

mee

ting

s th

at f

ocus

on

stud

ent

achi

evem

ent

and

impr

ovin

g it

, ver

sus

facu

lty

mee

ting

s th

at t

alk

abou

t co

pier

s an

d th

ings

like

tha

t al

l of

the

tim

e.”

(Elm

focu

s gr

oup)

“And

, Bru

ce h

as b

een,

rea

lly, a

god

send

. He

chal

leng

es m

e an

d he

pu

shes

me,

and

he

the

mon

thly

mee

ting

s th

at w

e ha

ve w

ith

him

her

e at

sc

hool

—th

e ad

min

istr

ativ

e te

am [

AL

T]

here

hav

e be

en w

onde

rful

. B

ecau

se h

e pu

shes

you

, and

he

chal

leng

es y

ou, a

nd h

e ju

st s

uppo

rts

you

all t

he w

ay. S

o, w

e—I

don’

t th

ink

we

wou

ld b

e as

far

as

we

are

if it

w

eren

’t f

or, f

or t

he G

etti

ng R

esul

ts t

eam

.” (

Oak

pri

ncip

al in

terv

iew

)…

whe

n sh

e to

ok o

ver

as P

rinc

ipal

GL

T m

eeti

ng ti

me

was

som

ethi

ng

that

tea

cher

s m

ight

or

mig

ht n

ot h

ave

atte

nded

, ser

ved

as c

ompl

aint

ti

me…

get

ting

tea

cher

s to

att

end

was

a f

irst

ste

p &

acc

ompl

ishm

ent.

G

etti

ng t

hem

to

star

t ta

lkin

g ab

out

inst

ruct

ion

was

/is a

noth

er s

tep.

(F

ir P

rinc

ipal

inte

rvie

w)

Tea

cher

s p

rovi

ded

lead

ersh

ip v

ia A

LT

Ass

essm

ents

use

d t

o in

form

in

stru

ctio

n“I

thi

nk t

hat

AA

LT

, the

tea

m …

the

y ha

d a

lot

of le

ader

ship

thi

s ye

ar. A

nd I

kno

w t

hey

felt

the

pre

ssur

e. A

nd I

thi

nk t

hey

wer

e th

e le

ader

s be

caus

e th

ey w

ere

the

ones

who

wer

e ke

pt p

ushi

ng u

s …

le

adin

g th

e m

eeti

ngs

… Y

es a

nd k

eepi

ng u

s fo

cuse

d …

” (P

ine

focu

s gr

oup)

“No.

Tha

t’s

neve

r ha

ppen

ed b

efor

e. A

t th

e be

ginn

ing,

tha

t w

as a

lso

a to

ugh

nut

to c

rack

bec

ause

peo

ple

didn

’t w

ant

to b

ring

stu

dent

—“I

’m

not

goin

g to

bri

ng m

y st

uden

t w

ork

beca

use

then

peo

ple

will

thi

nk I

’m

not a

goo

d te

ache

r”. …

it to

ok a

whi

le to

get

aw

ay fr

om th

at …

to c

hang

e th

e m

inds

et …


able

8.

(Con

tinue

d)

We

star

ted

the

Aca

dem

ic A

chie

vem

ent

Tea

m. A

nd t

hat’

s be

com

e ou

r cu

rric

ular

lead

ersh

ip, u

h, t

ool …

It’

s be

en a

cha

lleng

e to

, um

, br

ing

teac

hers

act

ivel

y in

to a

lead

ersh

ip r

ole

beca

use,

um

, the

y’re

ve

ry u

ncom

fort

able

tel

ling

thei

r co

lleag

ues

som

ethi

ng t

hey

don’

t w

ant

to h

ear

… w

hen

it c

omes

to

som

ethi

ng t

hat

they

’ve

com

e to

a

conc

lusi

on n

eeds

to

be d

one

that

’s n

ot g

oing

to

go d

own

wel

l, th

ey

still

wan

t th

e bo

ss t

o te

ll th

em”

[lau

ghs]

. (F

ir p

rinc

ipal

inte

rvie

w)

Bec

ause

tea

cher

s w

ere

very

unc

omfo

rtab

le w

ith

shar

ing

and

brin

ging

w

ork

in. N

ow it

’s n

ot a

n is

sue

anym

ore

… W

e’ve

nev

er s

at d

own

wit

h co

lleag

ues

to a

naly

se s

tude

nt w

ork

… A

nd w

e ne

ver

got

toge

ther

to

talk

. A

nd w

e ne

ver

got

toge

ther

to,

to

plan

tog

ethe

r. I

mea

n, it

’s ju

st t

hese

last

tw

o to

thr

ee y

ears

has

rea

lly c

hang

ed t

hing

s …

(O

ak p

rinc

ipal

inte

rvie

w)

“I r

emem

ber

my

sub

days

wer

e ju

st t

o gr

ade

the

asse

ssm

ents

… a

ctua

lly

got

me

to g

rade

the

ass

essm

ent

… r

eally

, rea

lly h

elpe

d …

and

dis

cuss

ed

the

resu

lts

… m

ade

note

s of

stu

dent

nee

ds o

n th

e sp

ot, s

o th

en w

hen

we

had

our

next

mee

ting

we

… s

et g

oals

… A

nd n

ot o

nly

asse

ssin

g bu

t lo

okin

g at

tho

se …

hel

ping

gui

de o

ur in

stru

ctio

n w

here

our

wea

knes

ses

are

… w

here

we

can

focu

s …

and

how

we

can

impr

ove

the

wea

knes

ses

and

cont

inue

wit

h th

e st

reng

ths

… m

ost

usef

ul is

the

wri

ting

ass

essm

ent

that

we

do a

t the

beg

inni

ng o

f the

yea

r, m

iddl

e, li

ke s

he s

aid,

and

the

end

of t

he y

ear.

It

help

s us

fin

d ou

t w

hat

need

s, u

h, w

e ne

ed t

o ad

dres

s w

ith

the

stud

ents

. And

I t

hink

it r

eally

hel

ps. [

Rig

ht a

way

.] [

Yea

h.]

Whe

reas

no

t, “

Oh,

sch

ool’s

end

ing

and

whe

re a

re, w

hat

are

we

lack

ing”

. (P

ine

focu

s gr

oup)


Frequent classroom observations of teacher–student interactions would haveprovided greater insights about linkages between GR Model elements, GR-relatedprocesses, and teaching and learning at GR schools. Likewise, observing teachingand learning, in vivo, in classrooms at comparison schools would have further illumi-nated our findings. Additionally, findings reported here are subject to the benefitsand limitations of using a sole external evaluator, as opposed to an evaluation team.

Implications for School Reform

Findings indicated that although implementation levels varied somewhat across GRsites, the GR Model was successfully implemented at GR sites. Moreover, GRimplementation established apparently unique processes that contributed toimproved academic achievement at low-achieving schools. GR schools also wererated higher than comparison schools on key elements of school improvement.These findings have important implications for school reform.

First, findings from this external evaluation contribute to the body of literaturethat supports the efficacy of change elements in the GR Model—elements that arenot unique to the GR Model. These elements are, indeed, evident in other reformdesigns (Fullan, 2000; Marzano & Kendall, 1996; Schmoker, 1996), although wewould note the GR model stands in contrast to much more prescriptive reforms thatspecify curricula and instruction (Borman et al., 2003; Desimone, 2002) and thathave shown the strongest evidence of impact. Nonetheless, the evaluation findingsreported here and in Saunders (2003) support the premise that goals (or standards),indicators (or assessments), assistance (or collaboration and professional develop-ment), leadership, and settings are influential levers for change. We have not yetestablished empirically the individual contribution of each element to improvedachievement. In one sense, the individual contribution of each element is irrelevant.The premise of the model is that no one element is sufficient; all are necessary. Wedo know that, in combination, and given reasonable development (i.e., Level 3 onthe GR Model Rubric), these elements were associated with more rapid gains inacademic achievement at GR schools, compared with the gains at demographicallysimilar comparison schools, and the overall school district, over a 5-year period.

Second, data collected in this study indicated that GR change elements were, atleast, somewhat evident at comparison schools. We believe that the challenge is notinstalling these elements, but developing and utilising them effectively and at a highlevel (e.g., a 3 or 4 on the GR Model Rubric). Recall that most GR schools were ratedat rubric levels 3 and 3 to 4 for each element, and most comparison schools were ratedat 2 and 2 to 3. If this school district reflects reform efforts in the U.S., then manyschools, based solely on local, state, and national emphases (e.g., content standards,annual assessment, professional development, school governance), and without localassistance of an entity like the Getting Results Network, might function at Level 2 orslightly higher (see GR Model Rubric, Table 2). In other words, the following arelikely to occur: (a) achievement goals are probably identified in the form of state ordistrict content standards; (b) school-wide indicators, in the form of standardised


tests, are probably used, at a minimum, to examine annual achievement trends; (c)assistance or professional development is likely to be available, at least intermittently;(d) leadership is most probably focused on improving academic achievement, at leastin general terms; and (e) settings, such as leadership teams, grade-level meetings, andfaculty meetings, are available and used, to some degree, to foster school improve-ment efforts. The GR Model provides one example of how schools might move thesechange elements toward a higher level of development and utilisation. We would thenexpect this to result in achievement benefits beyond those currently observed inschools where standards, mandated tests, professional development, and otherreform features have been mandated from the top down and with less systematicattention to change elements.

Third, findings from this study, like countless other school reform studies, pointto the critical role of leadership. In particular, this study’s findings seem to demon-strate the promise of increasingly distributed leadership that involves both adminis-trators and teachers working together. Indeed, some prominent scholars have calledfor more creative forms of distributed leadership that not only improve studentachievement, but also do so in socially just ways (Hargreaves & Fink, 2006), and inways that transform the culture of learning and teaching in schools (Fullan, 2002).GR implementation required teachers to assume academic leadership roles (RQ 3,Finding 5), helped establish tighter linkages between administrators and teachers(RQ 2, Finding 2), and brought about change by challenging and assisting teachersto actually provide leadership to staff and GLTs (RQ 5, Finding 2). Not unexpect-edly, some teachers in GR and comparison schools expressed concerns about theirroles as change agents (e.g., worrying about the extent to which the focus onacademic achievement in reading and mathematics impacted other aspects ofschooling, including physical education and health, music, and art). Mostprominent among those factors that helped schools successfully implement the GRModel was the combined leadership of school administrators and teachers via theALT (RQ 4, Finding 1). We address, here, three implications of our findings forresearch on, and practice of, leadership as a lever for school change.

First, we conceptualise and focus on leadership as it relates to other elements ofreform and school effectiveness; that is, not leadership per se, but leadership as itbears on setting and sharing goals or standards, utilising indicators or assessmentinformation, seeking out and cultivating assistance and collaboration, and establish-ing and maintaining productive settings such as leadership teams, grade-level meet-ings, and faculty meetings. We could have written a very different version of thisarticle, one that might have isolated our findings on leadership. But it seems morerelevant and theoretically more important to examine and discuss leadership in thecontext of the full GR Model, wherein leadership functions as one of five highlyinterdependent change elements. At least as it is conceptualised in the GR Modeland enacted at GR schools, leadership operates as a necessary although notsufficient condition for productive change.

Second, related to our first point, the findings of this study provide one example ofleadership as it functions within the context of a specific, and fairly concrete,


approach to school change. Although the GR Model/Network is curriculum-free(unlike, e.g., Success for All) (Slavin & Madden, 2006) and process-oriented, it hasadvanced over the years to become increasingly “nuts-and-bolts” oriented in itsapproach to the process of change and to the administrator and teacher leadershipthat successful execution requires. For example, GR assistance and trainings forprincipals and teacher-leaders, for the most part, do not focus on leadership per se.We certainly highlight the concept of “leadership that supports and pressures”—oneway to distill the large body of empirical and theoretical work on leadership. And wework with teachers to help them understand principles of keeping meetings focusedand people on-task (e.g., by creating and adhering to agendas, and dealing withproblematic personalities). But, more commonly, GR assistance and training forprincipals and teacher-leaders focus very directly on leading and facilitating specificand concrete instructional tasks. These tasks include, for example, using a specificprotocol to review and interpret standards; applying detailed procedures to adminis-ter, score, tally, and analyse results of specific assessments at the BME of the schoolyear; and using a specific protocol to identify common student needs, formulateobjectives, and analyse student work in the context of grade-level meetings.

An ongoing debate in the reform literature weighs the comparative benefits anddrawbacks of more conceptual versus more “nuts and bolts” approaches to reform(Bodilly, 1998). As the GR Network has evolved, we have consistently found it valu-able to try to work on both planes with school staffs—on the concepts or principlesunderlying concrete procedures and the successful execution of those procedures.Without question, over time we have produced higher levels of implementation andeffectiveness, and, we would argue, stronger and more effective leadership from bothprincipals and teachers as the nuts and bolts of the work have been made increas-ingly clear. It is possible that our efforts have been myopic—that we have cultivatedhighly contextualised, task-specific leadership skills with limited generalisability. Wehave completed no studies to investigate this possibility. Perhaps this is one of manyissues that merit study. Nonetheless, while noting this caution, GR results to datesuggest that benefits do accrue when we approach leadership programs and inter-ventions from a more contextualised, task-specific orientation. In other words, whenwe help educators in the schools get things done. Moreover, findings from recentmulti-method studies of primary school reform are consistent with our experiences.For example, Lithwood, Jantzi, and McElheron-Hopkins (2006) found that schoolleadership and school improvement processes accounted for the largest proportionof variance in explaining modest yet significant differences in student achievementacross primary schools.

Third, it seems important to note what the GR Model and its implementationdoes and does not focus on in terms of leadership. At the centre of the GR radar isteaching and learning—not school governance per se. Inherent in our initial theoret-ical orientation of school change, this heart-of-the-matter focus has been furtherreinforced as our work in schools evolved. Originally, we non-descriptly named ourmodel the “School Change Model” because we saw it as a vehicle for makingfundamental changes in how teachers approach the work of teaching and learning.


As the focus on student outcomes became increasingly pronounced, and the educa-tional system as a whole bore down on tangible evidence of student achievement asan impetus of improvement (e.g., Schmoker, 1996), we started to use the name“Getting Results”. Throughout the years, we have tried consistently to stay as closeas possible to the major source of improved achievement over which schools haveenormous direct control—teaching and learning in the classroom (Wang, Haertel, &Walberg, 1993). Some GR schools, as well as non-GR comparison schools, havehad governance training (e.g., LEARN, a local school-district reform effort that waspopular in the 1980s and 1990s, and School-Based Management), and some havenot. But we have not found focusing on governance issues independent of the issuesthat most directly impact teaching and learning in classrooms to be an effectiveapproach to accomplishing our fundamental goal: improving student achievement.

We raise this issue of governance in light of the strong findings this study uncov-ered about distributed leadership. Perhaps teaching and learning, in general—andmodel elements such as academically focused goals, indicators, and assistance, morespecifically—provide a fruitful common ground upon which shared or distributedleadership among administrators and teachers can be cultivated efficiently andsuccessfully. This is not to negate the potential import of shared or distributed lead-ership that focuses on governance. However, governance has for many years receivedsubstantial attention and emphasis in the reform literature. GR results suggestsubstantial promise in shared leadership that focuses specifically on helping teachersget results in their classrooms.

In the GR project, strong external assistance provided by GR staff helped admin-istrators and teachers become more effective leaders and experts—professionals whofocused their efforts and who pressured, supported, and assisted colleagues towardachieving specific, measurable goals in ways that impacted teaching and learning ona daily basis. GR staff provided the expertise initially, to help implement key aspectsof the operational model—ALTs, grade-level teams, collection and scoring ofstudent papers—and subsequently, to help sustain these settings and foster expertiseand leadership among key educators at GR schools (e.g., teachers who served asGLT leaders and ALT representatives, principals, academic coaches who workedacross grade levels). In time, a critical mass of focused educators emerged at GRschools and teaching became more coherent across each school. One very consistentrelated theme was that GR staff provided assistance in ways that earned educators’respect and impacted their attitudes and teaching practices. This assistance, whichoriginally was perceived by educators to be from an outsider, morphed into a moreinternal form of assistance as educators began to interact with GR staff as “one ofour own”.

Paths that lead to getting results are not likely to be smooth. Indeed, GR schoolsand comparison schools experienced many bumps in their respective paths,including a gauntlet of emerging demands that often impeded their way. Externalexpertise, combined with leadership within the school, helped schools stay on track.Elements of change could wane—focus in the form of goals that are set and shared,indicators that measure progress toward achieving goals, and settings where the nuts


and bolts of change must occur in order to get results. Indeed, these elementsseemed to be in danger of continual derailment since so much effort had to beexpended to maintain them. The external assister played a crucial role in helpingschool staffs keep focused on their ultimate goals and maintain settings and activitiesthat were essential for achieving them.

References

American Institutes for Research. (1999). An educators’ guide to schoolwide reform. Washington,DC: Author.

August, D., & Hakuta, K. (Eds.). (1997). Improving school for language minority children: A researchagenda. Washington, DC: National Academy Press.

Berends, M., Kirby, S., Naftel, S., & McKelvey, C. (2001). Implementation and performance in NewAmerican schools: Three years into scale-up (MR-1145-EDU). Santa Monica, CA: Rand.

Blase, J. (1987). Dimensions of effective school leadership: The teacher’s perspective. AmericanEducational Research Journal, 24, 589–610.

Bliss, J., Firestone, W., & Richards, C. (Eds.). (1991). Rethinking effective schools: Research andpractice. Englewood Cliffs, NJ: Prentice Hall.

Bodilly, S. J. (1998). Lessons from the New American Schools’ scale-up phase: Prospects for bringingdesign to multiple schools. Santa Monica, CA: RAND.

Borman, G., Hewes, G., Overman, L., & Brown, S. (2003). Comprehensive school reform andachievement: A meta-analysis. Review of Educational Research, 73, 125–230.

Brophy, J., & Good, T. (1986). Teacher behavior and student achievement. In M. Wittrock (Ed.),Handbook of research on teaching (3rd ed., pp. 328–375). New York: Macmillan.

California State Department of Education (n.d.). Content standards for California public schools.Retrieved March 4, 2002, from http://www.cde.ca.gov/standards.

Carter, T., & Chatfield, M. (1986). Effective bilingual schools: Implications for policy andpractice. American Journal of Education, 95, 200–232.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches.

Thousand Oaks, CA: Sage.Darling-Hammond, L. (1997). The right to learn: A blueprint for creating schools that work. San

Francisco, CA: Jossey-Bass.Denzin, N. (1978). The research act: A theoretical introduction to sociological methods. New York:

McGraw-Hill.Desimone, L. (2002). How can comprehensive school reform models be successfully

implemented? Review of Educational Research, 72, 433–479.Earl, L. M., & Lee, L. E. (1998). Evaluation of the Manitoba School Improvement Program.

International Centre for Educational Change at OISE/UT.Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership, 37(1), 15–27.Fashola, O., Slavin, R., Calderón, M., & Durán, R. (1996). Effective programs for Latino students in

elementary and middle schools. Unpublished manuscript. Baltimore, MA: The Johns HopkinsUniversity.

Fullan, M. (1993). Change forces: Probing the depths of educational reform. London: Falmer.Fullan, M. (2000). The three stories of educational reform. Phi Delta Kappan, 81, 581–584.Fullan, M. (2002). The change leader. Educational Leadership, 59(8), 16–20.Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. Chicago: Aldine.Goffman, E. (1959). The presentation of self in everyday life. Garden City, NJ: Doubleday.Goldenberg, C. (2004). Successful school change: Creating settings to improve teaching and learning.

New York: Teachers College Press.


Goldenberg, C., & Gallimore, R. (1991). Changing teaching takes more than a one-shot work-shop. Educational Leadership, 49(3), 69–72.

Goldenberg, C., Saunders, W., & Gallimore, R. (1996). Settings for change: A practical model linkingrhetoric and action to improve achievement of diverse students. Proposal submitted to the SpencerFoundation.

Goldenberg, C., & Sullivan, J. (1994). Making change happen in a language minority school: A searchfor coherence (Educational Practice Report No. 13). Washington, DC: National Center forResearch on Cultural Diversity and Second Language Learning.

Greene, J. C., & Lee, J. H. (2006). Quieting educational reform … with educational reform.American Journal of Evaluation, 27, 337–352.

Harcourt Assessment. (1996). Harcourt educational measurement. Retrieved from http://harcour-tassessment.com.

Hargreaves, A., & Fink, D. (2006). The ripple effect. Educational Leadership, 63(8), 16–20.Joyce, B., & Showers, B. (1983). Power in staff development through research on training. Alexandria,

VA: Association for Supervision and Curriculum Development.Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. Thousand Oaks,

CA: Sage.Lieberman, A. (1988a). Expanding the leadership team. Educational Leadership, 45(5), 4–8.Lieberman, A. (1988b). Teachers and principals: Turf, tension, and new tasks. Phi Delta Kappan,

69, 648–653.Lincoln, E., & Guba, Y. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.Lithwood, K., Jantzi, D., & McElheron-Hopkins, C. (2006). The development and testing of a

school improvement model. School Effectiveness and School Improvement, 17, 441–464.Little, J. (1982). Norms for collegiality and experimentation: Workplace conditions of school

success. American Educational Research Journal, 19, 325–340.Manning, K. (1997). Authenticity in constructivist inquiry: Methodological considerations

without prescription. Qualitative Inquiry, 3(1), 93–115.Marzano, R., & Kendall, J. (1996). Designing standards-based districts, schools, and classrooms.

Alexandria, VA: Association for Supervision and Curriculum Development.McDougall, D. (2002). Getting Results project external evaluation. Long Beach, CA: Getting Results

Network.Merriam, S. B. (1998). Qualitative research and case study applications in education. San Franciso,

CA: Josey-Bass.Miles, M. (1983). Unraveling the mystery of institutionalization. Educational Leadership, 41(3),

14–19.Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.Peterson, K., & Lezotte, L. (1991). New directions in the effective schools movement. In J. Bliss,

W. Firestone, & C. Richards (Eds.), Rethinking effective schools (pp. 128–137). EnglewoodCliffs, NJ: Prenctice Hall.

Rosenholtz, S. (1991). Teachers’ workplace: The social organization of schools. New York: TeachersCollege Press.

Sarason, S. (1972). The creation of settings and the future societies. San Francisco, CA: Jossey-Bass.Saunders, W. (2003). “Getting Results”: Final report of achievement impact. Long Beach, CA:

California State University.Saunders, W., O’Brien, G., Marcelletti, D., Hassenstab, K., Saldivar, E., & Goldenberg, C.

(2001). Getting the most out of school-based professional development. In P. Schmidt & P.Mosenthal (Eds.), Reconceptualizing literacy in the new age of multiculturalism and pluralism(pp. 289–320). Greenwich, CT: IAP.

Schmoker, M. (1996). Results: The key to continuous school improvement. Alexandria, VA:Association for Supervision and Curriculum Development.

Slavin, R., & Fashola, O. (1998). Show me the evidence! Proven and promising programs for America’sschools. Thousand Oaks, CA: Corwin.


Slavin, R. E, & Madden, N. A. (2006). Success for all/roots and wings: 2006 summary of research onachievement outcomes. Baltimore, MD: John Hopkins University.

Taylor, S. J., & Bogden, R. (1998). Introduction to qualitative research methods. New York: JohnWiley.

Tharp, R., & Gallimore, R. (1988). Rousing minds to life. Cambridge, England: Cambridge Univer-sity Press.

Wang, M., Haertel, G., & Walberg, H. (1993). Toward a knowledge base for school learning.Review of Educational Research, 63, 249–294.

inside the black box of school reform: explaining the how ... file52 d. mcdougall et al. wave of...

Documents