pocumm resume - eric · pocumm resume. se 019. 543-fournier, raymond h.-development ,,and...
TRANSCRIPT
ED 116.911
AUTHOR"TITLE
PUB DATE-NOTE
S
pocumm RESUME
SE 019 543-
Fournier, Raymond H.- Development ,,and Implementation of Computer±zedMonitoring System in Mathematics Grades 4, 5, 6.Apr 75 4,
144p.; Submitted in partial fulfillment of the"requirements for the degree Of Doctoi of Educationd r'Nova University; occasional marginal legibility inexamples.used -- 4
EDRS PRICE if-$0.76 HC-$6.97 Plus Postage .1
DESCRIPTORS *Computer Oriented Programs;'Computers; CurriculuM;Elementaii Edgcation; *Elementary School Mathematics;Instruction;.Mathematics Education; *Objectives;*Schedulingl *Testing .
ABHIRfCT 1
A computerized monitdring system was developed forgrides 4-6. Objec;tites and corresponding test items were stored oncomputer. Instructors selected objectives for each monitoring period.Stdents were tested frequently using interchangeable forms of, testscovering these objectives. Tests were computeg scored and interpre d,
by teachers. These in.6rpreqtions led to'-curriculum-decisions aswell as student feedback. After the system wad implemented, studentPerformance on both norm-referenced and criterion;. referenced testsimproved. (SD)
r
***************************************************,******************** Documents acquired by ERIC,include'many informal unpublished ,
* Werials-tot a vailable from other sources. ERIC makes every effort *31c to obtain.the besticopy,available: Nevertheless, items of marginal *
*-reproducibility are often encountered and this affects, the duality *
*'of the microfiche and hardcopy-reproduptions ERIC makes available *.* via the ERIC Document Reproduction Service (EDRS). DRS is not* responsle foe the quality of the original document. Reproductions *.4* supplied by EDRS are the best that can .be made from. the original.*******************vc********************************16o****************
A PAU.S. OERTMENTVF WEALTH:EDUCATION &WELFARE ,NATIONAL INSTITUTE,OF
EDUCATION
THIS DOCUMENT HAS BEEN REPRDDUCED EXACTLY AS RECEIVED FROMTHPERSON OR ORGANIZATION ORIGIN-ATING IT POINTS OF VIEW OR OPINIONSSTATED DO NOT NECUSARILY REPRESENT OFFICIAL NATIONAL INSTITUTE OFEDUCATION POSITION OR POLICY
a
VELOPMENT AND IMPLEMEAAT
COMPUTERIZED MONITORING SYSTEM
MATHEMATICS GRADES 4, 5, 6
by RAYMOND H. FOURNIER
SubMitted in partial fulfillment the requirementsfor the degree of'Doct4c:of Educ -tion, Nova 1.1niversity.
Old Westbury Clu er Maxi I and II
Dr. J. Bprum, oordinator.. April '.- 1975
Ale .rr.
S
TABLE OF:CbNTENTS
Abstract iii
introductioh iv
Acknowledgement
.1. Needs1
11. The Problem 14
The Reports 36
IV. Support Materials .54
V, How it Works
Vt. jaciston Making and Introduction 65,
yi i. Developing a Mastery TestingSystem *. , 0 71
VIII. Implementation
IX. Evaluation 94
X. Budge't
4
Self- Assessment 132
BibliOgraphg.' 134
Appendix 135.
A
TABLE,OF/FIGURES
#1. ObjectiveBank- sample page
° #2r .item sample page -.23
#3 Math. Monitoring Test - sample page, 25a
#4 , Student,Answer,Sheet 27
45:- Form used for enteringchangingor deleting objectives 28
Form uSed foreniering changingor deleting test items 30
#7 Form used for entering, changing oraltering inforMation about.hestudent h ,Program,
#8 Form for generating. computer listingof teachers in Orograk 33
#9 Form for indicating test items usedand position occupied on each test 35
'110 Monitoring system report for thestudents 37
Monitortng,System report to teacherlisting-class lierformance. by
objective 39
#12 Monftoring system report to teacherlisting individual performance bytotal test -41
oilItoringsYstem report for prin-cipal listing student performanceof total level each objective
.
#14 Monitoring_ system report for_ Central,Administration listing stutlentforniance on each objective for allstudent in school dlttricVat each-level 44
4
4
.4
TABLE OF FIGURES pontindiiid...;.
#15 Monitoring System Report - 14sting4indiVidua1'students and performanceon each objective
#16 Computer print-out of currentstudent bank
#17 !tam Analysis-
#18 Closs',Reference GOide Listing in-struttional materials by objectiveS1'
#19 Answer Keys
#20 Mastery TeSt - sample page.
#21 Class Performance Chart
#22 Chart illustrating relationshipsfor Project Director
#23 Chart illustrating relationships. ,
for Team Leaders
#24 Item. Analysis - Math
e
'425 Error Analysis - individual studentperfermence by. 9bjective
1.
1
45
47.
, 49
56
.57
:60
61
78
79 .
96
97
P
ABSTRACT
The purpose of this m was to deVelop and implementA
a eoMputertzed.monitoring systeM f6r Math grades 4,:5,'and 6,
(The same sysieM, once established, can be uSed,f9r monitoring
R aditigas well) 10 three yeafs. The hypothesis is that such ano.
impl entation will Improve student performance.: both om norm-
refere ed and criterion- .referenced tests. The practicum was.'
successf lly'egecuted and the hypothesis was' confirmed,
"41
iii
6
J
INTRODUCTION
This report will:detail a three year labor fOr the purpose.
of .helping Students perforM better academicallyt ,--11;e Needs will{
discuss-0d context prior to the establishment of thisInonitOring
system. Section Lly. the. Problem will detail theconflict betweenP
norm - referenced and criterion - referenced forMs of education. The
Offerent views of constitutes learning should be made evident.
Section III will-describe the Brentwood, New'York-apPlicaticin of
Comprehensive Achievement Monitoring., The procedures; forms and
systems.. will be explained. In Section IV:there is a discussion
and explanation of the...material that needed).to be develop4d to
I
help parents, students and teachers implement and interpret the9
system. Section V will detail the sequences of activities re-
quired.foruse of the= monitorin system. Section VI will deal
primarilyl,with the oUtcomes.Of he prqoess4 that is, the use of
the Information to modify behavior\ materials andzalirse content
to help students improve perf.rmance. Section VII reports on the
back-op systehicti provi4es\ tailed information oneach objective.
,Ih,Section'Vtli the course of implementation of thii program in the
.DrentwOOd Schools is narrated. In Section IX the results in terms
of student performance are repohed. Section X discuSses_the budget
from the beginning,of the prog-rSm to the present date.
v
4.
ACKNOWLEDGEMENT
This project and the report that follows were made possible by
the People of Brentwood, the Members of the:Board Of Education
4
John J..-Curio, President.%Maureen Belanger, Vice President
.ismael ColonJoseph. J.4Curto
Anthony.Fericio'ISabelGutentigMichael T. Scanlon
the Superintendent of Schools,-Mr, G. Guy_DiPtetro, The Assistant
SOpetiPtendet of SchoOls, Dr. Arthur R. Brieger, the
teachers, team leders.andaides whb have all `contributedtoward the
success and improvement of the program. ,However, there 'is one part
in the.CurriculumOffice where all materials, Forms, Procedures and
.
people c oss. At this point, the management byMes: Hilda Forrest
kept every hing moving in the appropriate direction and-contributed
immeasurabl to the success of the program.
Mr; Robe t :Howlett, Assistant tuperintendent of the Half Hollow
Mills School di trict andAlfis staff have chosen to work out the
primary program w lob constitutes the first half of an elementary
curriculum. Mr. Ho lett's cooperation in proposal development and
hiS responsibilitysiexecutingHhis half are worthy of commendation.
41,
0
!.0
.
NX. Needs. Brentwood is a'commpnity'in the middle of tong Island,
Prl'ocx.0 tit was sparsely:settled. The extension of the Long
tsland'Railroad plds the population ekPlOsion a the'1950's Ptis:1 ..
1960's brOught lWid development. By 1970: theee-were more. than
60,000 people in thecommunity and 22,000'students in grades K712.'4'74'
Twenty large schools had been built and yet the secondary schools
were on sp14,-,session. ,The population of the school dlsteiCti
was reported to the. state, was 23% Spanish surnamed,: 5% black and
82%'wh1 te. The *Community devefked as, housing Projects which were
'occupled as quickly as they Could be bait% Some Markers commute
:io Or'city, nearly fifty miles awey: However, most commute to. other
icommunities-much closet. Brentwood has not developed extensive in-. ,
dustry of 'its own.4
.
Student performance On -etaodardized tests indicated ,a narrow
top band of achieliement and wide band of sub norm performance.' In
1970, on California-ACIVe ent tests and New York State Tests, it
seemed 'clear that the rend had developed: to a critical state.
1970 PEP Tests
% Below Minimum Competence,
'Grade Reading liMath
3 24 19
6* 31 .32
The educatimar.fundamentals of math and reading were not being suc-\
:cessfully taught: .SucCess was not possible in our secondary schools
8
`. .1.
-2-Ilp
The Assfstent'Superihtendent thismriter met extensively
io develop a_ strategy fo? reversing theldisasterous trends revealea-, .
,jrthe norm referenced test icores.
kseries OfAax `irtreases. and bud et rejections had forced a., 4
depletion of the:sUPerVtsjonstaff I he BrehtWood Schools. Where
there had once been a full. time MatheMatics COordlneX or and staff.
lend a full time Reading Coordinator and staff, there were now none. ,
left:... An oUtcOme-WaS4:hat'An this period there was a decreasing. ,,,
.,
ampUnt of supervislon,and:Monaltoring.of.both the mathematics and. y.. ,
,r, ead i n g programs. it was very clear,frahrecent budget votest hat.. .
there Was no chanee of hiring personnel to conduct supervision of
the program. Asve studied our.problem and consulted other school. ;
diStricts, the New York State 4ducatton peOartmentand local uniVer-. .., ... _
sities, another.p6tnt of view slowly began to domiriate our thinking.
, .y _ ,
it teemed that new methods of monitoring instruction were beginning,, N.,
\
.to evolve. k
.General procedures, of Monitoring1
Step 1. Specify the overall performance objectiVeS.tO be
: accomplished through a given eduCational'Program.
The performance objectives would complete the arst
task for accountability.
! Educational Accountability: ,A formatfor Monitoring the'Teaching-.1;earning YProdess* TerrD. Cornell', EPIC Diversified,Systems Corporation. Educational innovations Press, TUcson,-Arizona, 1971.
10'
Step `Specify Oe time Interval for monitoring. The instruc
tors that. involved in the instructional program
shOuld aOree,on the inter.Valtof time that will Oass,
between monitoring points. /In most instanCes4.0e
monitoring period would probably be weekly or bi-
weekly..
Step 3. Select a person to be responsible for keeping a rec-
-cord of each instructor's sheets for each monitoring
p6int. This person's responsibility mould lie to Make
sure that each sheet is completely filled out - state-
ment of objectives'that will be dealt with during the477-
'monitoring interval, completion of Planned Program
section at the beginning.of'the.monitoring interval,
completion of Actual."Program section at the end of,
. . .
themonitoring tnterval, indication of what'objectives.
were met and what.objeCtives were not-Met..
In addition to general procedures, there are.sOme tndi
:C
vidual tristructor\procedures which should be followed f this
nitoring system is to provide relevant fedback. The'proce-.
Aures are listed. in sequential order.
Individcil Instructor Procedures
Step 1. Prepare interim behavioral objectives in a sequential.
fr
order of accomplishment as they relate to the over-.,
, all performance obJective(s). The;only difference
a
fr
I
-4-
:between interim objectives and the overall-per-
lormance objective is that .there will be a shorter
time interval ar.rlici the situation 'under which the
behavior will be obServed will be more specific:.
Step 2. Select those objectives that will be considered for the
Step 3.
given monitoring period..
.
Complete the Planned Program as it relates to the
aCcomplistiment of.the objectives identified in Stip 2
above.
`Step 4. At the end of the monitoring- interval, complete the
Actual Program section and indicate which objectives.
. `.were completed and which Objectives were not completed.
Step 5'.." Recycle by se i g the'next obj,ective(s) as
: ;
4 Tied in Step above.apd.complete the Planned Program.,
section on the next. monitv109
Step 6. Continue process.So
Almost completely from necessity through the)ack.or alter-
natives that we could afford, we were headed inAhe monitoring .., .--
. .
.
.
. .
.
: .
.
prograMdevelopment. Though there were some common elements 16
,.. . , .
monitoring systems, objectives, items-apd detailed records, therep...
.'
were many directions that coul.d have been purSued. \
, .
We had long felt uncomfortable working with standa\rdized.tesis
bto evaluate our student progress. Lilie school administrators'
4 '
everywhere, we ,knew' there were inherent weaknesses 'in' standardized
.
12
,
-L
05X-
.
°
tests. Yet,, ttietr level of icceptance ks-so,high that they gen-
ertly receive -full unquestioned confidence of the tax -payer de.-'
Otte their obvious limitations. Norm referenCed tests may be.
Characterized-fairly by the following qtialities: .
.A norm referenced. test. compares the performante of one In-
dividual against the performance of tang other individuals. The
.point at whiCh half of the group scores abode and the other 4alf
,scores below is the "norm ". Tests. are'.-standardized so that half
. of ...the students will score abov'a certain point and half belbw.
When a situation, arises In'which more people sciirleabove than'below, for example, a 'test might have to be re-normed. In order
rto achieve this standardization items chosen for, the test tend, to
be instruction probf.. That IS. if an .item is taught successfuly,
to 4th graderstall over the country, it 'Will not make a good testiiin for a norm referenCed test becausemore than half Of the
population will score -correctly.
Oependence on. these teaching proof itema may Fjave contributed
to another phenomenon associated. with norm, refereticed testing.- One
can prRatct with emazi ng, accuracy what a group of students will
,score on norm referenced tests _if one knows the following. infor-.4.
mationt
-a. income'f father
b. 'language ability of mother
c. social economid position n' the commun I ty
4.
-6-
For whatever reasons, inthe United States the children in
-wealthy commuditriS attainhighscores on standardized tests and
the children In poor...communities attain loW.scores on'Standard.7
!zed tests. This phenomenon has the status ofjegal acceptancei
In NeW York State.:The standardized N\ew York Statetests'admin7
jttered in grades 3, .6 and.5;for mathematics and reading serve' as..,-
. ,.
AhelUndaMental. information -ln a booklet of Performance,ExpeCta-, .
tionS ,that the State Departme cpublished for every-sdhool district.
Infdimatlon'concerning the dis ict's wealth (per pupil valuation)
A other socio- economic data are then combined to predict-.what
g
stoonts will score on subsequent standardized tests. in:additfon4
in 1974 the legiSiature Passed an act called Chlter 241 of Edr:
cation Laws of New York State. This lawprovidet..thet a,dittriCtw
will be reimbursed in state aid at the ratio'of 1.25 for the per-.
centage of,studentsscoring.two more grades below level according
to the New York State Test. The system it alMost infallible.,.,----,
Wealthy districts have high scores and poor diStrJetsave low
Scores. -A, survey of the literature finds no notable large scale
exceptions. All of this is prefudtto:the generalization that)
while we are requiredto a4Minister norm referenced tests, those
Who administer'theM,generally khowrthat they will be ofilttle use
in organizing the instructional program.. On norm referenced tests,
50% of the population must be in 'the sub norm positiorr that is
eratiy interpreted as failure in our society.
14
-7-
Criterion referenced tests.on the other hand% are based on.
I
behavioral'objectIvespeeato and'precise ttftements'of expec-.
tation for the performance of the learner after the specified con-..
dItions,have'been net Attainment of the objective under those
specifiedconditions constitutes "meeting., the criterion ". c0-!.
terion referencecrtests measure the,nerfkMance of an indlyidual
student against the performance specified in the objective.
Another way of d'epressing the Idea, is to '§ay, that criterion
tests measure a student against the cur Itulum instead of coMparing.
his Performance against other compares a students per-
formance against the objective. it does not compare Students to
each_other,or.:to a norm.
Another facet orcriteridort resting is that thetests,:contain
OnlY.Itemsthat test the stated objective they are directly related
td the inStruCticna1 program.
In addition, criterion referencedqestt provide feedback con-
cerning the instructional program. Since .the objective stated
Student performance after certain conditions were met Unsuctessful
performai-ice may mew'''-
a. %further study and practice needed by student
b. prior learnings need: to take place
c conditions stateein'the objective are insufficient
or inappropriate to bring about the.student's. behavior.
Whatever the result the information given by the test is always useful
ft
to examine student performance o(r the means of instruction or the
test itself.
A ,
Interpretation of c .1 erioh refere ed teSts:alSo varies
-4
greatly from that ofl.nor .referenced tests where a StUdeht gener-`
. .
ally passes or falls. For criterion'tests, hIS'achlevement-is a
matching of hie perforMance against the 041 stated In the objective.
Failing_:meens that he has not .-.Yet performed as expected; passing
means.that he is ready for another. Criterion tests are
locally developed to meet the needs of local Students. They may be
adjusted actOdingjito needs or according to student performance.4
They are flexible -- related to students, their, goals:and-their
needs. Since criterion tests: test the objectives dIrettly. these
.:tests are very 'important 'in evalliatIng the instructional program.
PUrpose.for
testin
Placement
g v .ReqUtred Capability
:.--
Test WhichMeets TheRequirement
To test retention of knowledge andski 1 ls: prerequ is ite to specs f tc, ob-
jectives to be .taught in currentcourses for'purpose of early rete-.vant placement.- Must pinpoint ..
speci f it -weakneSses.
x :CRT . NRT.
Weadmi drst ratl on
of "final test(s)"from prerequisitecourse (s).
Pretest To test .entry knowledge and skillsof specitic, behav ioral objectiveito establish entrrknowledge dataand detect students who alreadyhave the required knowledge andskill.
x 'CRT , *NRT
An equivalent formof the "final test"of the turrentcciurse.
,ca
16
x CRT NRTIA test' composed ofrat vat. terminalor subtermlnal
tterbi,
a studentil'exitcomparison, item bytry'knadlidge Inermine the individualprovementin specific.ing .the course: itemsecifid behavioral
x CRT -,'NRT
An equivalent formtif:tne pretest
t
'To determt e how a student, or CRT ,x NRTgroup of s udents, rate on a scale. A test composed offrom high o in relation to items sampling aall studen s on a nation-wide( subject matter de-!basis, in. a subject area. Rating signed to separateis plotted,omthe "bell ;shaped" students by abiltty-curve, or'knoWredge.
_9_
PlacementADUringCurrentCourse)
,:x CRT i NRTA7test;_romdose'd of
relevant enablingtest ftemt.
To -pinpoint spec tic- weaknesses-; -
particularly Whierarchicalsituations, in-'order td--preScri4e
learningmaterialsiand'Situationsdesigne4.til eliMin te weaknesses,and. strengthen re entlon. 'Items.
'must test attainm nt of specifidbehavioral objet iyeS,
MasteryTest
To deterMine a tud4nts' masteryof a specific s bject or operation,or a major hie ,,rchital sub-unit.May determine astry of pre7requisite know;edges and skillsor terminal O. jecOve0..,
. Itemsmust test spe Hit behavtOralobjectives.
Posttest TO deters tneknowledge initem, withorder to destudents' isubjetts'dumust test sobjectives.
NationalRanking':
AptitudeTest
. .
To sample tudent's aptitudes or=abilities in an effort to predict(on a pert ntile scale) those whowill do well and those who willdo poorly in.a particular disciplane or j b.
CRT x NRTA test of itemsthat has shown pre -dictable validityand reliability.
-17
AssigningGrades
.4 -10-'A
To separate studtntp into a seriesof categories from good (400. topoor (F). The subject mattersamOled. Emphasis is quite.oftenon testing in an effort to separate"A" students tom "F" students.
"CAT x.-NRT--
A test composedOf ttems samplingpreviopsty taughtsubject Matter.
In 1971 Dr. Brieger and I met with Dr. Robert O'Reilly Chief
of the Bureau of Research' and Cultliral Affairs, New York State Ed-,
Ucatron.Department.and Dr. William Gorth School of Education,
UniVersity. of MasteChusetts.:At t1.11$ meeting we learned of a_co0-
pdterizecl monitoring system Wf4h they-,hadeveloped.with D.
_ . . .
.
in.Celifornia: The Computer-offered the ingredients needed for our
(This will be disaised\in'detall:in the .`section -:on
Process of Implementation). That is, large-scale capacity for per-1
scoring' ;.and cleriCal functions with great speed.fOr over
night tUrn7aroUneof resultSJor teachers.
The. eqUirements of, the Computer, hciever, were
objectives, items;'-tests answers, etc. had to
and exattl ey detailed ".in. preset form and:sequence.
that inputs,
be precisely
lt.becaMe this
writers fask to organize a staff of principals and teachers
a. .write and select objectives
b. develop test items for, those objectives
C. develop tests
to
2 instructional Module Criterion Referenced' Testing, New YorkState Education Department,. Bureau of Research, Albthy, 1571.
\18
.
d% develop manUais for teachers,, parents and students:
e. develop cross reference guides whiCh indicated where -.in- .
the materials available in our sChoOls, a particular Ob-
jective was taught.
f. code objectives. for the-computer
g. arrange for programming
h. prepare teachers
I. distribute all required materials-
j, coordinate test-ingt,collection,'scoring.and return
k. monitoring of the monitortng.prOgram to evaluate its
contribotiorito the educational prograM
it was determined, that the first trial of the prOgram would be
in grades 4, 5 and 6 in mathematics. Mathematics was chosed be-.
cause
. we were
matics.
, . .
great need to improve our teaching of math
b. more work had been done with behavioral objecpyes and, .
testtems in mathematics than for other subjects:
the precision of- mathematics seemed to. offer greatermpe
for success in the first trial.
:Having elected to employ a crtterTorrreferended testing system
,to monitor student progress in the Brentwo4 Sthools,..we were given.
the opportunity to re-examine our instructional program in terms of
needs of the students and community:, Re did. not pursue. this in a
fashion
-12-
that Would haA'satisfied Dr. Stfflebeam, yet We did ex;
.amine quite carefully where we stood in relatiOn to What it was we
hoped-to aChieve-.! Some.of the factors, we considered
.r1. CommUnitY Expectations. Brentwood parentP,
observed, believed in education as a way of
financial and social standing for their children. They
had not, yetlread professpr jenc10.4 3
were:,
it,was soon
improving
. Student CharacterisOcs. AtadeiliicallylItentWOOdentS,
followed a statewidpatern. lrdgrWe performance was
high bUt decltned at a faster than statewide rate at 6th
:gradeend 9th grade level. An immediate goal became the
on 6th grade tests to atleast)
im irbyemen of performance
StAeWide levels.
St deWinterests. There is a strong,faith..p edUcationLi
on ithdpart of `primary grade studentsP: They believe they
cart iear and that if they learn they will succeed. This
faith4iiikinishes with age but remains strong.
Perforie.-Characteristics of Graduates.
25%go to fouryear 'colleges
3.-,45%10 on to two year colleges and,voCationalS
c. -50%,JOIn the work. force
-FamilY-immediateiY
training
military or marry to raise a
3 Cristopher Jencks, et al., Inequality A Reassessment.of theEffect of Family and Schooling in America, Basic Books, Inc,
.Pub., New York, 1172.
\N,
-.3-
,
/
t, In rexiewing community and;student needs it became clear that x1 r .
there was a strong desire-and need to perform well academically.
..'.,
-..-
The Olemma occurred because the, community standard fo,r judging(
succsS Was a norm-referepted test --'The New York State Test
t, .administered at &adfs 3, 6, and 9. The results of this test are
,
annually published in the local. neWspap r. CombiunItits al-ea4:11-:--%./:..,
listed and easily matched against each o her.
In addition to ,improved student learning.deMonstrated on
Criterilon tests, we would have. to adigeve improved scores on *Stand-
" . Iardized tests at the 6th grade level,. 14 made this a three year
goal of'the program.-
in
21
The Program.ti
At an early meeting 1 heard Dr. William Gorth discuss the
problem of developing useful 'information abot student'PerfOrm00
as one otthe central, issues in curriciUluM development and adjust....1
ment. He posed the question in terms of graphs.
Student Score
100'
Sept. Jan., June
in this graph we see illustrated the. situat t Most o
.us hope for when we teach. Many teacherS'belleve it is what
actually happens. :rills, of Course is the ideal tearning4pattern
in which a student.comes to class-not knowing what is in: the. Course.
At the starting. point'in September, the teacher teaches.this par-.
ticular un,14',. The student proCeeds to go from near zero'not know-
ing
.
to lde4loo knowing. Furthermore, he does not forget. He keeps
.:on knowing, overtime. This probably doesn't occur regularly in in-.
strUction but most teacherS are notin a position of knowing- They
dO not have a pretest to Measure:.whit,the.student knew on entering
the course. They do not havean objectives based test to measure
with precision an objective of the course and-they do not have.
22
157 i
folta4-up tests on that same objective for measuring retention
overt-40e.
Another Pi:IsSibi iy
*IfudentSCOre A
50.
O
Sept. Jan, June
in-'thll."little bit at `a time" learning:the StUdent learns,,
40 time the teacher teaches. doos. not fOrpet4nd the in-
Crements are even. -Again*, thts-ts probably not a regmlar. pattern.
Even so it would,require 0 pretest* teSt ancrretention test pat.-
tern to know that this Is occurring.
Student Score
. 100
50
* 0Sept. Jan. June
in this pattern the studeitit-came into'tho course .knowing thiS
23
-16- ..b
.. r ,Jpart. .He'did not forget it and regular. instruction Had :no',effect.
,
. .
With pretestAnformatiOn and objectivelfoiowup tests the teacher'
woUld be :in a position to - o o tciteaching,Something else- to a r
student who exhibited such racteristics.:.
Student' Score
100
50
0
a
A
Sept. Jan. June
in this..patiern a student enters not ,knowingthe.objective. He
is taught.and learns tt very weI14 However at th4 next testing he
scares near,zero again. This pattern indicates forgetting. This
systerriCof testing indicates that re-teaching is now necessary.
StUdent Score.
100.
50
Sept. .Jan. June
In this pattern, one that every teacher has probably felt, the
24
-17- .
student enters the class knowing the objective. The teacher teaches
for this objective and the student promptly becomes confused. This
situation it probably inevitable .in every class. It really poses
no problem unless the teacher, does not gain the information that
the student is now confuseci. This requires the pretest testdand re-.
test. pattern.'
NOnelofipis is new. EXperts ilvevaluationhavejOng preached
it as away of knowing the effects of our instruction. The problem
has always been: that
a. .stating the objective
b. making_ the tests
c. scoring tests
d. Writing reports for the-student and other
interested in 'his progress
have been too time consuming and too comPlex -0, be done regularly
by an individual, teacher..
Comprehensive Achievement Monitoring.* whIch was outlined for
'Us by Dr. O'Reilly.And pr.-apeo,- was a systemjor employing the
computer to keep track of objectives and ,student performance,
Write reports, to keep cumulative infothatiOn from testto,test-.
for individual objectives or for total test.'
In the follOWing Toa4e I-will attempt to describe and explain.
the Comprehensive Achievement Monitorin4 System that was developed
in OrenttiNfood. ln.curschoal district we (in the happy American
. 87
A 4.
,tradition of creitinaraceronyms for everything)7, called it B.E.SkT:
Brentwood Educational ,§ystem for Teiting..
The components of.the.system are:'
1% Behavioral Ojectives
2. TeSt Item bank
3. Test construction program
,It. Six teat forma". forfi.each leyel for ,each. semester
5. Answer keys
, 6. Cross 'reference guides. i
,
7 Manuals for students, parents, visitors
8.- Forms for entering-information into the vielous .
dal'a\banks:
.a. 06jectiVe entry form
b. .item entryform
c.,, student update' - add and
d. test'construdtiOn4orm , ;,
.
..,.,
Administrators and-teacher& tO our district (sec6
:' . P.
PleMentation Will !6fplain'prOcess) seiectedian wr6te.,
objectives appropriate for our students for gedeS,At:'
Eight basic strands viere:developed:for,the program:
1. Numbers VNuMeratiOn
2.: Basic Operations With Naiural Numbers
1. Rational Numbers
4. Decimals
tiro ors im-
'behavial
5;
26 ,
For.
.ftg.
)7
Number SentericeS
6. Vord PrObles
° Measurement
Aeometry.
-1-9
r
Of-these topics discrete objectives7Were written. (see
,in:orde to keap track of the-objeCtivesand toy .have
) outer manipulate them a coding system had to be developed. A
simple procedure was developed.
01 00 .100 OpMath Grade ToPic Specific Objective-
so that objective-40-1-0k-02 ol ..MeanS
-a. math objective
b. fodrth grade'.
c. second topic (basic operations...with naturallnumbert)
d. ffrst.Objective
For this monitoring system a bank with algnimum'of four test
items is required for each objective. The qdality of these test,
. .
items can be determined at the end of a. semeste(or year through
de.611edTiteM.analyits supplied by the computer. However, in-the
first writing of test itemSthe followlhgTirocedure:helps. toassdre
that items will reasonably test the objectivet they OurpOrtto-test.
. -
a.. A good objective has-implicite a test item Oicluded.
Example: OBJECTIVE -'The student will demonstrate the
-2
04 -02- 006 -60 MultipricatiOn
OBJECTIVE: Students wi I,- select the nature 1bumber which isthe correct answer to..a :given multi pit cation =problem.
The multiplication problem.may 'be of two types:1) the multiplicand is a 3-digit number and the
mul+ipl ler is a 2rdigit number)..."'
2/ :the multiplier and inultipl icand.are both 27digftnumbers.
EXAMPLE: 253X. 32
04- 02.01 O0. Divislot)
RATIONAL NUMBERS
...04-03-00100 Equivalent frac+Ions
(A) 9096
(C) 8096
(B) 7090-
. (D) 8090..
Second, semester-
_ .
OBjECTlyE: Student$ wil r.l' select the correct' answer a,givendivision prOblem,of natural- numbers; The"diviso may have one Or 'two .digits, and 'thedividend wit, have, four or fewer. digits.
EXAMPLE:- 15)3926
(A)
(C): 65 'RI t
26ta.
(p) 261 _Rt1
Second:Amester
OBJECTIVE: Students will select the proper fraction that Isnot- e9uiva lent. The denomibators of° a II
equivalent fractions will be less than or. equal
t9
EXAMPLE: Which fraction i Is not equivalent to
(A) 4 .(8) 5
(C). 8 (D) 10,20
(fig. 1) ObjeCtive Bank
28
-21-
ability to count to
TEST ITEM - 4, 5,. 6,
(a) 5
(b) 7
(c) 8
(d) 9
If the test item closely ref=lects the objectiVe is- 4
Ilkely-a suitable item.
b. Reading analysis is sometimes
propri atness of
needed.to'determine do-
I. vocabulary in--the itep
27 sentence length in the item6'
17 sentence camplextty In the item.'.
-c. In the multiple choice format. the quality Of(the distract-
ors is an important part of-the quality of the test item.
The two most common problems with distrectors are
1. ;11.1e distractor is so .close to being the correct
answer that it may not discriminate between a
simple student error' and a lack of understanding.
'2. The distractor is so obvious that lt.coUld not
possibly be considered as a suitable answer by.
any student.
The Multiple choice format was chosen for the testing
program because it most easily lends itself to computer-
29.
-22-
ized testing and scoring. In a. well constructed' item In
mathemattcs, the chances of a student'S guessing the
correct answer is-25%. However, keepidg In mind that the
student will be tested srx times in each semester we can
figure the guessing odds another way. The chances that a
student will guess the bOrwct answer on two tests in a
row are .0625 which ts,a very: smalli.ohance indeed.: (see
fig. 2)r.
Nati.tetly it was necessary. to develop a coding system. to enter
items in the coMputer bank, items were coded as follows;
00 00 40. ..00* 00
Math Arade-, Topic ' Objective
order to.make a monitiartng Systei-11 from a:bank.ofobjectives and
a bank of test items a test construction progrark is necessary.
System decided upon for Brentwood has been to first develop. the
.
The
trend, testing system to be fotlowed and supported two years later
by a mastery tasting system..
The monitoring system initially designed to' take advantage of
the computer's speed and capacity for storing and reporting on large
numbers of variables. In addition, the CAM -system makes it possible
to'test students frequently, ona,,targe number of objectives while=
using a limited number of t st items. There are a number of reasons
that this is desirable, On _example is.the length' of the test. f The
monitoring system which os six tests per semester, will have six
different test forms. If for test items are available for ead'''
04t,02-006-00.MultiplAcation4' `
OBJECTIVE :. nts will select the natural number which is 7:4the orrect answer to.a given multiplication problem.
:The ultiplication problem. may be of two types:.
he multiplicand is a 3-digit number and theultiptiler is a 2i-digit number;
2) he Multiplier and.multiplicand are both 2-digit
253-x 32
(A) 096
(C) 096
umbers.
(B) 7090
(D) 8090
Second semester
04-02-007-00 DivrSion
OBJECTIVE: Stu enfs will select the. correct' answer tO a
giv n division problem of natural numbers. Thediv 'sor may have one or two di its, and theOs/to:lend 'Will have four or fewp, digits.
?< EXAMPLE: 15 3926
(A 265 R12
( )65 111.1
(B) 261.,R5
0)-261 .RI-1
sec-of d semester
RATIONAL- NUMBERS
04-03-004-00 Equivalent fracti
,OBJECTIV ;, Students will select the proper fraction that .4q
not equ i va 1 ent. The dencthinators of all --) \equivalent fraCtions,.0,11 be lesi than or equalto 100.
1. EXAMPLE: Which fraction i Is not equivalent to I ?2-
(A) 4 (B) 5
(C) (D) 10T6- is= -25
(f
objective, each-test will test 4/6of the objeCtives. Stated
another way over. a-course of six,testsan indiVidual student will
be tested four times on 'each objective. The whole class will be
tested on. all of the objectives. After the,second test all students
in the'cllass.will -have been tested at least one time on each
ectivp seliCted.
4 The computer program for assigning students to tests works
this way._
1. There will be six testS'.
2. There will be'six forms for each test to, "be sure:that
students do not a forWof the test MOre.than one
time and to assure that eah'student4Will take all the
test formsa table:is ctevellope.d. First six groups are. k
'created 1h)each classroom. Then they arOaisigned test
forms by: computer he following -manner:
Group lW - Tests 2 3. 4 '5 6 1
Group Igsts 3 6-1 2'
Group 311f -
.
'Tests', 5 6 2. 3:-
Group 4 - Tests 5 6 2 3 4.,
Group 5 - "Tests. 6 1 3 '4 5 .,,
Group 6 - Tests 4 5 6.
There are many advantages to this kin. sampling. First,,
.
' it practically-eliminates the problem of cheating On tests. Since, ,
students cannot. be'sure that a neighbor has the same test form,4.
2
-75-
there is little'point In cheating. Secondly,the tests can be of
reasonable duration% If all s,tus. s were tested on all objectives'
in eacktest.the test would, this case,,necesparily,bt 1/3 longer.
The teStsAghich are given twe ve times a year; last from 30 to 40
minutes. Economy in'itime required is important when there is such
a test, frequency.
The computer will assign student groups to each test, form; it
will also/assign objectives and test items dn each of the six test
fo1-ms. (See Appendix G, N.Y.S.E.D. Cylol format)
The .resulting six test forms in multiple choice format, (See
Appendix H) are then available for skudent use The following kinds
of question tVe fig. 3). There would be 25 on'each test that would
. be responded to. This is one of .the two kinds of testing systeMs
Aeveloped within the BEST Program.'.Thedefinition of objectives
by grade or difficul)ty_level, the computerized random assignMent
Hof test forms for the purpose of sampling student performance regu7
larly on the total set of objectives; this is the Trend test aspect
of the peograrn. "'A complete set of the Trend tests in the BEST
system may be seen in appendix H.
Mastery tests are different in that one objective'is tested
five or more times on one test to establish a level of mastery;.'
such as five outf five,,four out of five,etc. The Brentwood
set of mastery tests may be seen in appendix R.
33
10
-25a
'LB 27 OZ.LB .1. OZ*
A) '3 LB. 14 0 . CB) 4 LB. 1 OZ. (C) LBS. 10 OZ. (DX,
ONE.SCHOOL.BUS.1CAN CARRY 47 STUDENTS. ,HOW MANY STUDENTS CAN7 SCHOOL BUSES CARRY?
c;CA)`, CB.) 289 co 329 co.: 747
a
IF JOHN BOUGHT 213 'JEW ,BEANS EACH DAY, HOW MANY JELLY BEANS WOULD HE0HAVE IN 67 DAYS?
,.CA) 271,.2. 14,161 (q) 271 (D) :A0161
*UCH FRACTION IS NOT EQUIVALENT TO
(C) 1232
34
(D) 6
IF r (fig. 3)
Math Monitoring Testsample page
-26-
A class of 30 students would have groUps of five students
taking each of thestx test forms* The student will have a pre-
printed answer sheet.- (see fig. 4) .ACross'the top is listed
the studendnumiler (astigned by census department'wheh student,:,
enrolls), student(hame, school number, grade and homeroom number.
Theteacher is MS0 designated .by number, usualtythe Same as the
homeroom number.
Spacesare,provided-for the,student to mark the answershe
-believes to bp correct. A regular number pencil- is used.: Our
first.-attempttrequireb use of a grease pencil. Somehow, we could1,
neverheve- 'enough of these on hand when they were-needed. This,
prObteMOiseppeared when:theA)rograM was_alAilged:to.use ordinary.,
pencils.
The bottom of-the page lists the 'Objective numbers for
the course. that this sttident is currently enrolled in The teacher.
marks or has the students mark the numbert-ofthe objectives that
have beep taught since the last trend test.' .
The entire procedure of coding for the comOuter was handled
in the curriculum office: The well defined procedure has evolved
the foLlowing,Alresently used forMs,,which organize information-
for the key punch operators at the data processing Office:
1. Obj ective Bank Updating Form (see fig. 5)O
The action box offers three possiblities 0 = delete,
1 = new, meaning to add this objective, and 2 = change,
4 35I 1
BRENTWOOD EDLIdATIONAL:SYTEM FOR TESTING,ais.
I S 6. 3
.5 11 3
-1;
1-.1 ; ir ; a i $ 3 ; S k1 ;:THIS SECTION FOR DATA PROCESSING USE ONLY. DO hoiPt/T ANY M4RKS IN THIS SECTION
3 A
2 3 ; 5 M n ;
3'.12nET NO j SNOW NAM( SC.f GRACE ...WM EAC.Eg WO' ACE DEC .20 UST PO
11271t411CLgMENT JOSEPH , 8 4 .000 406 i G11 1 114202
T 1 STUDENT ANSWER'tECTION. INSTRUCTIONS FOR FILLING CUT THIS. SECTION.1. CHOOSt-CNLY ONE :.:.;*X FOR EACH QUESTION. --2. USE A,NO. 2 PENCIL TO MARK THE BOX OF YOUR CHOICE.3.- SHOULD YOU WISH TO CHANGE YOUR CHOICE. COMPLETELY ERASE THE DLD.MARK.
A 9 C IC
A V C 4
13 10,.,
4 I ..
A 9 C 0 A a F51 4 12 1
'a E. c D A
/ 1 311'
.i
A P 7 A
7 14 1
A C A 9
8 - 1511 C. 0
.4 9i /7
A 's18 7
19A
0 A
20S2
21 1
B
1
p
e,
22 jc
_ 29A e c C' A
23 r 30A 9 C 0
24.
25
26
27
2
A 3 C C.
A IS C 0
A S
qA
31A :-
32A 9 0
33A e
34' . .
.2. A B.
35
B w a A a
36 -
3? A` e's.
38A
& 3 -39 . .
0 a A f43 50 ,
A 5 c A e :44 .. 51.: .
!45 . -52
t16A 3 0 % A
40 . 47-P
RT 2. OBJECTIVE SECTION. INSTRUCTIONS FOR FLLING OUT THIS SECTION.
1. LISTED BELOW ARE THE OBJECTIVES FOR THE CURRENT SEMESTER2. USING A NO..2 PENCIL. MARK THE BOX LABELED YES FOR EACHOBJECTIVE TAUGHT SIR.CLIEEPIESEIV=EILELLSLP -
3. SHOULD YOU WISH TD CHANGE A MARK. COMPLETELY ERASE THE OLD MARK.
CSJECTIVEh.J_MBER
14201004
14232004
1423300/
14233005
14235002
14236005
14236010
14237004 ..,
TAUGHTYES NO
r.!
3
OBJECTIVE TAUGHTNUMBER YES NO
14201006 nvi nJ - U
14202005
14203002
14203006.
14205003
14206006
14207003
.14208002
[1
0
2
0
1.
r, r
II
OBJECTIVE TAUGHTNUMBER : YES. NO
14201007
14202006
. .14203003
14203007
14206001
14206008
14207004.
14208003
:.;
C
(fig: it) C 5
Student. Answer Sfiffetn
A:
V.--
Er:
A
A C
OBJECTIVENUMBER
142.01008-. ,
,14202007
14203004
14205001
14206004
'14206009
14207005
36
TAUGHTYES P.O
-
1U
.
O.-It03
.MlD
C.
111
rt CD
'0.;
=to' .1%
O nm
.t11nr (D<
=M
.Ill
U:1
111011111111111111111Msam
mum
unumm
unnumm
unnUMMUMMUNIMMOMMOOMMINUMNOMNISMOINUMOOMMOSOMMIUMMOISM
1. 111111
11111111111111111111111111
1111111511111115
011101
MIMIN10111111111111110110111111111121111111111110111
51 11111115511111111111111111111111110111111111111
3.3115
1011111111011 111011111011111111011111111111111110111111101
1 11111111111111111111111150111111111111101111111111111
MOMMIOURIN MOM=
NINIIIIIIIMMONINONI11111111111111111101 1
11
I-11
11
11
11
11 11 1
11
11
1
OMM111111115111111111111 MOM 111111111301111111111111
ION
111111111111111111131111151:
annimonnum
n nunIMMOOMMOOMMOMONMIUMM MOM OROMMOMMOMMUM 1111 MINUMMOMMI
MOMUMMIN REMNOMMOOMMUMMOMMUMUMMU
ENUMMIMMUMNO
MSOMINUMMOSIBUMMOSONSIOS- MMUMNIMISOMMUMMUOINIUMMINME
11111111111111111 1111111111
ED
COD.
ED
111111111111111011111111111 1111111111111111111111111111111111
MOIMMOMOMOIUM11111=111111111111111111111 IIMMIIN1111111011111111111
nu ansimansinnum
mun
.nnunnuninnumn
L111111.1
iMONO OOMOOMMIMMUN 921222NOMMUNECUMEME
1221WO
MirO
PTPPT
TIT
E22kkiE
ldN
MI
NOMMO
MOON 001=
v
tirmn21-,
..J7417,,AIEEPFP1143=17
9-G7.1411FR
'01.61WN
iri9975IFfiTi:W
it1977.FFFIRiT
iTiqi ffivii.F.1;11;14T
+11-43
Val
1222U
9102
0
0°N
111Y.jM
a."13
3 ? jAU
ki
u.uo 6u1,10pdn fuee eApefcio
tt
1I
cQ
' M »1.1*A
wl
Is1 .0
ST
l
slot. s
UZ
I
131,11
144
141841!*41rIgrivoilisume 1st %
ILI*
asigaisi imorortas /A
tmaspubow
,094 anyt sapnbas bast144.1* tail 14114 "II
4114414414
1i 1141 pi *obi os u atito v p.
11041 41144440 4'140311,3~11
Paul, ay1%
aqetons A
M iha 1.1v4 .01.4..0
'110811.4040. 444,03/4134 8../{
14 acota >dot .0.4.4.1A
1 NA
Asti,
ty )4hot QM
. $44P1 Al 4.444 e)ft p
41wew
4410414 b.411,u3
vat,. opoi a,,11 on414.414
aw.fer &
pulp pv1 so ausao 15.41opp H
owe 464 pirsisa '1.;11/
time
ea 1.14 p4e101411411.111104 eow
'Aspire° tiv O
ti1DhY
X3 /1
1P4101401It
41,1544ro 1.411..otio4
'Ow
lsA
y 316.4
VO
W*
,a. 01.1p
3A1121110. W
I 31311011
tow 4%
0,4 lit to.01110.411/10 to i11 1141 O
no' 1101"latiu+
,1 at salsasihrquoikausf 14141:44. m
eta!Susi
41146151I;t1.444411,:n It
IMO
Mud() sal
*Oft44.11.4143 14
-0.4341110 14 I430101 41153
tppat* .g ilina
*eat11
*40103 11'444py
op..%* W
u.; itII/t113111101.13111
011.11.w.0)11-6114-61:M
ail4l
44.434.4.11
4/4.0111
101,13v
O121
meaning only the change inditated in the text is to be
made. The objective nuMber was previously discussed. In
',the objective number the area refers to math, reading or
other subject. In Brentwood math is always 1. 'Level
means grade 'level or difficu ty idvel: Topic refers to:
,,..A.
l
one of the eight'Strands of the program. Numberj-dentit
fies the position of the object in the sequerice of objects
forthistopic. Four boxes for sub - topics 'have not been
used the elementaiw program up to this point, The
source box'Proyides for a code:.to identify the source of
the object. 1 stands for Brentwbod, 2Tstapdslor New York
State Bducation Depatment, etc, The next tWoboXeyden-.AS
tify the number of-lines Of text in this objective. The
text of the objective is then written, in, one box for
each letter space, punctuation, etc. Thi orm.has been
widely used in our reading prOgram. it does no t-provide
for mathematical symbols and and geometric figures. Until
we can code those figures for the computer
have to be drawn by hand and stored on paper.
2. iteaBank Updating Form (see fig. 61_
Thjs form f011owsessentialiy the same procedure
adding items, deleting items and changing items.
Storing objectives and items is half of the storage bank
problem The'other half is reCording.ana keeping current
for,
.38
SUMMOMMUMME MIN
1111111111111111MU1ll11 ll
mummummummumummumen
inummumummummo
IMOMMOO
NEM ONOINIUNIIIMIONNINIMMUMMION5111=11111111
NUMMOMMONSOMIMMININIONONOMMONOMOMMONIUMUMMOIMUMBIANOR
1ri
MIUMMUMMUMOINUM
MOMMIMINOMMINUMMOMMOOMMOMMUNIUMMINOM
uiiiiunuiiiiiuuuiiiiibiuiiiiiiiiiiiaiii
11111111111
OMMINIUMUMONION
INIOUN
IMINOINSOUNI IMI
MANOINIVININ 40 LIMMO
MN OMNI
VIN
14 ilkof to -v4 $30,413 tittityriassodsit pits,0,04
NI mho
1.1=49 pit
4414,1441.611.11.,
twyU
Ul',IN
+A
watlutO
atlam
13001 NI tat
arse 1131Aint3a11fttlia atueam
Pitze11304, ay. &
sloe A 104.1-.0
via &aw
e igit so,lt+A
tu. 13414.) /01ilisi11'444414 tw
at, w $449140 ll tit
Ittrutru tutu '341..11113 pJilaW
ny .114;Pa1gO.N
4 Slito3 N .
1.4421, spfxon 2z, it 11.3it
1107111+3".1 t M
ia pug 4A4lt V
at
.44 114.4 spiv 01114,A pkvi-ispos..
AIN
" 1111 eu.vep1i4,41 t 11-1 IndiiiqU
OIS' SO
Pta.130011,N
CIT
CA
tiV9N
I9t4VH
211
-*413011.1,141/Z
.4 SUM
.
-he also! iorantwow
r fl.r entssurA"O
P V111400
-4.11042 K l I T
iri!sacliativ w
att aql 311443
taltcr41 emllespa
#4u3w
pe+apon: to
solul titrflii N
V i1317C
1 V
OMMUMM NUM
1012 IN MN
summummummummummun mumnommum
wamonkomman js
1111111111111111111111 4.4 al fir
OW
UM
W211
MIN MOW Al
IMONINIUM
ME IWO 'WOO
w'4 at.
01110! 401 Itso lost Irm
o* A6,3 ist
.00,4 PI if
sin Supoolsemra ago apw
w. 4.1.14i N
ska
vines Was tql
$!,/u3*w
ail .41 if aro iif.0611./%1 allU
j0044 11.411 M
U] IP
'MO
WN
Will 14 aa$,13
Actoonv p.oalcO
Ap.A
.1.. 9431,4 p
it at lt ttot
antra.,M
A%
-31`
the millions of bits of information about s tudents,.
. Update Student Bank Form (see fig 7)
.-This form provides themeans to enter, delete and Change - .
data about students, in the programS. The first lineOf
iformation.identiffes the student In( name, number, school
and grade, it also pr:Ovides space to indi whether this
is an addition,
are available below because students `in Brentwood may po-'
.tentially be entered in four areas; for example: 1 - Math,
2 Reading, Spelling and 4 - Language Aets. Each of
these areas would heed informatio n to indidate
a. area (which subject)
b.- teacher
c. course
d. section
e. group
Each tes.t,,,period, the student-haS an opportunity to move
froth one ability'level to another. The teacher may change
deletion or change. Four identical blocks
the level at which the student is being tested by simply in
dioatin9 the new student. level for the test period coming.,
Teacher NameiForm.(see fig. 8) y
This form is-for coding.teacher names and numbers for,each
school.4
r
Teacher
3738
39
Teacher..
37
Course
StudentNo
,4
'69.
044
LA
ST. N
AM
E-
10.11
12.13'14
1516
1718
19;
LE
VE
LS
Test 1
.Test 2
:Te St 3
Test 4
'Dist 5
Test 6
4647'48
4950
51.5253
5455
5657
AR
EA
INFO
RM
AT
ION
'Sect
43144
!!!L
EV
EL
ST
est 1T
est-2T
est 3T
est 4T
est GT
est 6,
4647
4849
5051
.6265. Sa
20
FIR
ST
NA
ME
1:2E
22'24
2526
27; 2830
31
Teacher:
59
6061.
AtT
eacher
6960;
61
_AR
EA
INFO
Course
6263
64
Sect
6966
1C
C.
01 --;C
01 4-1-10 V
)4) .41
L.
rac
,O
4-14-1
.1..
to(0
O.C
1L
.(1-
osO
0C
tU
0) 0 a.C
:40) to .0co C
E 4.1
L.
o .c 0 cu. u
LE
VE
LS
Tost 1
Test 2
Test 3
Test 4
Test 5
Test 6
6869
70-7172
7374
75'76.77
78.79
AR
EA
INFO
RIV
IAT
ION
Sect
6566
LE
VE
LS
Test 1
Test 2
Test 3
Test
4T
est 6T
est 6'
9869:70
71...727344
7676177
'7,79,
CARD CODE. M3
Thither Number
2.3.
4.
6.
7.
8.
9.
11.
12.
13;14.
15.
16.
13,
'19.
21,
22.23.
24.
25.
23.
27.
2a.
29.
30.
31.
32.
34.
-33-TEACHER NAMES
1.
2.
3.
4.
5.
.6.
10.
11.
12.
13..
14.
15.
16.
17.t.
18.
19.
20.
21.
22.
23.
24.
25.-5
26.
27.
28.
29.
30.
31.
32.
33.
34.
5.
'SCHOOL No..
Teacher Name
4.
.4 (fig. Form for generating computerlisting of teachers in program,
4
Test Coristruction (tee f 9).
This indicates another _immediate product from operation of
the computer prograM: PtUdent inforniation produces -first
the answer sheet. Objectivec,item and test form informa-
tion produces a testing of ltmes in sequence astheY will
appear; on each test for-M., Once this form has been produced,
it is'a simple task to retrieve the item cards and set the\m
.up,in teseform. These are then photographed and the test
are produced.
171=1.
13
(f i g . 9) Form for;indicating test i terns used
and - posi t i on ocCupi ecr on
each t
-36-
The.Reports.' A test scheduleAs publishe prior to the
beginning of the. year: On the day of the test, the team leader and"
an aide and teacher diStribute ans'w.7ertletS" being very careful
each student has the precise test called for by the students com-
puter, pre-Ortnted answer sheet,. Students take the teit. The
arecolleeted,and. sent to the district curriculum office where they
are spot checked. The entire district's tests are then sentOto the
data:peoceSsing center,Where they are read by the orScan machine
which prOdOces key punched .cards which7pre-then processed by the
computer. By nine o'clock the following morning'the eeports'are
picked up and distripted to the school by the curriculum office.
The following report's are pro ed by the coMpUter in ;the
Btentwood Educational System for Testing:
r
1. Tierld Test Stbdent Repo see ftg. 10)
This report is designed- to be taken home by the student.
The infOrMationjs rather stralghtfOrwatd,. All the net-
essary inforMation concerning the student name number
and sectiOn, his teacher's number and school nUmber at
the left from top to bottom are listed, and the topics
and objective numbers.that were tes-th4l, The'stUdents had'
previously taken home the Manual for Students and Parents.
(see apPendix B), Using the manual, the sttldents and
Orents can find the specific Objective that*as tested..
.7he manual always gives a test item Samplewith each
r
't.REVA
TESTI
STuDEN1>ft-EP'0,RT
STUDENT NAME
: THANHAUSER
KERR/ ANN.
I.TEACHER NUMBER! 504-
COURSE NUM ;ER 1 000
'7"
STUDENT NO
713545
'
SUBJECT: MATH
DATE-1-0,4/15/75--
.6t
-
Tg'sr.Pc-nor, NUMBER
--
-.
TO
PIC M
AL
E'
.-,
, OBjECXIVES'7--- -1.7------2'".-"---3"--.
-'-'47----,r-
NUMBERS Al.0 NUMERXTIO4
iS201004
_,
-.
-
NUMBERS ANT) NUNERATIO1
.
..1520100-5
)
'
CT
.
'11UmBERS Aft
NumERATIO4/'
15201006
II.,
C'
- -.
, CT:.
.....- ......-
NUMBERS *HU NumFAATI
15201007
C _
-kurboas: A4bil
RATrost
Aspdlool
tit5ATIPM
NU
MB
ER
SNO.t.UME
04-
/52010011,:
+4
NATURAL NUMBER OPERATIONS 4520003
4NATURAL NUMBER GPERATIONS 15202004-.
:-
.--
RATICNAChUmtEPS
''RATIONAL NUMtER5
i.rRATIONAt. NUM-n145:
-
a RATIONAL.NUmeERS
RATIONAL NUP.BERS
,sRATIONAL NumsEAB
,
RATIONAL NUMBERS
h.
RATIONAL NUmBERS
!-RATiohAt. surBEAS
--
4RATIONAL NUMBERS,
.
": DECIo#ALS
' DECIMALS .0
" DECIMALS
" DECIMALS,
4DrcviaLt
"O
EC
IMA
C.S
I.
NUsWER4EuTENCES
1
.NU
MB
ER
ftBER bENTENtE8
jNUMBER SENTENCEV
' NUMBER SENTENCES
' kOPIrp.4.(NsLEMS
WORD - PROBLEMS
wvADWeaLEKS
.
°*ORO PROdLEmt
1,gRO PROBLEMS
r'
-
,WORD.ROOLEMS
''
MEASUREMENT
*GEOMETRY
: OEMiETRY
GEOMETRY
152p3por.
11n13002.
15203063
'CT
CT
CT
C---
-CT
CCT
CCT
-w w-----. -cr
CT
CT
15203004
-CT
-7"5203005-
WW
t,
15203006
t
152.03007
CT
152030092'.
m-
CT
11203010
'K
Eli
,W
15.203011
CW
1520!,001
N-
C
-15p04002
t-
W
1W003
1'5204004
15204005-
.16204008
15205005
.
11404006
-
10205007
15POS008
'
IRP06001
15206004
15006005-
V1:6206006
L
is?o600
ibn6o09
15207002
15208002
'
15e08003
WC
.1620W008
dry_
w L-.-
:..
...
SCORE
t CCORREtT4 wwRON0. W.440
RE
SP
ON
SE
,TxPOST/NSTRUCTIONAL
'CC
C'
.4
CCC
,
"
5L
i
1
m4.1 4-IN
C>
.4/1
01 4-1CL
.a)
.c
-O
0ti
rn
,
PERCENT COPRECT OF TOTAL
PERCp*-CORRECT OF POSTISTRUCTI0NAL
-
pARE6t,:commimTit
HtEst
PERIOD
NUMBER
e3
45
4!4'12
80
100 100.
=
-38-
jective staterent. In matching the, student reportWtth the
manual the student, teach and parent cove'objective by
objedtbieWobjedtiVe the student's performance on. the
test:of.the prevfOus day. The code at the bottom0 of thi
page indicatet that a
C means correct
means wrong
N means the student did not answer the:question
T means that thiS objective has been taught
Implications fordecision making Mill lie discussed later
The bottom ofthe student report also gives a percent score
on illthellems-and'anotherperOentscore for those test.
.
.
items. n mhich the teachers have indicated that teachingA
has taken place.
2.. Content Summary Report (see fig. 11)i
This report is designed to give the teacher information
about group performance on the objectives that were tested
down the left column the objectives:are listed. Again,
the teacher has ,a manual which is .a key for providing
statements of the objectives and sample test items. The
t
"figure under cOluMn A tells the number of,ceeonses that
were made for th: Objective. COluMn B gives the percent
correct= of thOIle responses. Column C lists the number of
.,
retponses on objectives that the teacher had indicated as
J.
470
..",.
.........*.......
,.......«...«..... ............ .44 4... . 0 .1.11
a&10 I
1,..*
. i ..... ' .**. ....
... two .... :..,
.. ...=
T R
EN
. 0 -T
E. $ T
::
T--
- S U
M M
A R
Y Y
R E
P O
R T
'-D
A4E
4. ---,
..4
'
...rit.
..........T
EA
CH
ER
NO
1 422 :......--.- ,...., ..-........ .... -,-,.........
..
.
411,...* . ..... *: 4.**** *... . Loy; .S
IJIIIEC
TI
.M
A, T
14 ............ .........
:-,--.-
sruotNr-G
RO
UP
._'.1.EV
EL.,-._,-- ....,-..----
---...-- ..
:-..-.
---
'-
..
.-I
-- -
- --'
- -=-
.
-
-.-
--.--: S
CH
OO
L;NOV.
03'..
..
..
..
............................................... ,...
........,
- .......-r,- .
.- - -.,...;
-.- -. --
,........._-TE
8T:P
ER
10,0,NU
M.
1...
08JEC
TIN
ES
A'
8,-
.A
--,B
.c0
' A_
-5
*C
.'04.
B...c.. --a
.
- A ,..".:..S
...-.
C -0:...
A$
-
-C0
...... -3, 4231004 - .`... . . 22.
68 . - .-.- . - 25- 40- --.... ,
, 26 46 ....-
...-.......-%
, ..:4 ova. 4.......,....:,.........., ............4 ........ ...............,.....i.1...! ,.., ...***:.It
.......14201007.---s.20.- 20 .--.............----18'. 4,,4:-:-...--..-.. 23.-.304., 14201608.
- 17' 7117
71 .....21 .57 ,..18 . 61 ...... 21..,52 ,. 20.,,
50.
14202004.
1532.
1932'
17?65.
.16
63-21
- 38 '1.8
39
.1420200517
7117
7119
; 84_18
8320
3019
79
-14202606 ----.1729
.1-729.-- 21. .67 ..--15---61.--- 21 -,67.-- 20-70 ,...
--4.).-s--.0 .1
-14202007.--.---24,.. 33- -. ...---:-.-.24.. 25------... --- 28.-43...z-.6
67-1*2.03001-- 19 - 32- --19 -32.-- 19 -.37-19- 37--- 20-.-4-p- 17
41.
;,..... 14203002 - 18.bi ..
18`. 61'-
.1963
..18..61 f..- 21.- 81_ 13
7'9-...:..-
-1420300320,
35,.,
'4-'100
.19
583 100
- 2157 .- -6
.67.-
...
1420300415. 95
1894
2190
-18.89
.
.:21'
9517
94
-1.4203045-_-...19
100 :....19.10b ...-- 17.- a/L.-A.6 - 88.--- 21.100 __.16 100 -
-
-14203006-----19 --63-- 6-- 83t,-20 -45-4-50-- 23-48--- 4 -.50
---14200047.--- 18 - 33 --19-33-- 20...55--17---59.
22.--45--20- 45« 4 14205001 -
..1872 ..
-18 . 7220
.8517 . 82
.. 22.-77 -26.75
'.-.." 'a
-a?
1420502.II
6819
6819
8418
8.3- .20
8516
81-
..
.--
.-"
...,tn Ili
-1420500319
95.19
95.19
10016.100
22 90115
95b
0.
11, L
l.
sn.,.14206004. -.-.-- 1974- -- 19 ..74...----18 -89 ---17-88'-22.-95
---19 . 95.'
.`"41.1420005___. 1850 ,. 12 .:33....-20 : 70 ...13 -4S
-........ 22-.77.-- 14.
71
."
...--
'...:::tu--.04°c"i 2---E
4-1.4t;e6001.---.20.
55 - - 667.- -. 20- 50.-- 3
. 33.---.22t145 .:.
6.
67 ,
-.14206006'.18
*418
k421
48 ,., 1953
...Z
O65,..,; 18
-61....._.
,......t.,.
., ....__
_.............._;__X
1
I. aa); ai
14206005ia
9516
9*17 100 T
11.100.21 100
44 100-.
142066t4919
6819
6821-
6219
63.21: 57.
1856
.r.
. 39.. 14
2919 . 47- -13. 38
2143
1540,
..,--14407003: --4-;- -19'
47 --.1
-.. 0 '..-,.-. 21.57 !--.1-100..:.-al....-.-48
'.
,..-.7.1.44.07004-. - '.1.1 a .5B
-1-f-:---.. .0.- - - ' 18
67 -1 --0 - - .23
-- 48 .. 15 -'50...'....,-,-,.-- -..----;--.91-.----1.---.....- ...-,,--...,..,..--,;.-''....-1.i......4
.,.,.'....4)..
.....:....
14207005.1-B
-96 -.-..
1-0
..20'40 -..- .-
..-.--.. '
,22-
55.15
.
..-40-,...-,..
'.,.-
4..--
'1;4207006..
20'_ 10 __
1926
...'
.-
--ell' 'as-
........
_-.-:.:
2 i -.'. F..71.......
-
1,9:79'
..-...-
.'..;...,.'., '22.:
86,
22 -..,:-.86......
...........,... ...
..o; ...,....
... ... a ,..... . . .. ..... I. ... in.&
. *., 4 - ..*ow
*........,
....**,...... ..6... ;
'am O
w ..la.. 11. I -,
Vp,a
.1 e 0 . . 4; , 4;0,0;11,1 ,...
0 .. -.» ....O
..
. I 4 A ....a *4,.........,..
....*.....i......*..... * ..,
* * ......******... ...........,.. -*. .., .sra......
1420$002.20.
4019' 58*
....-1420S003
.19
79
AT
CT
AL R
ES
PO
NS
ES
5 .v PE
RC
EN
T C
OR
RE
CT
OF
TO
TA
L -RE
SP
ON
SE
S- /A
l--: ..t.
...C.
aT
OT
AL
r'PO
ST
INS
TR
UC
TIO
NA
L RE
SP
ON
SE
S..
.
----0 I*. P
ER
CE
I-CO
RR
EC
T., O
F R
0ST
INS
TR
UC
TI O
NA
L -RE
SP
ON
SE
SIC
I.
............vlos ...a ..
. . %O
..
.,......
.... .
*......
..... . .
.. 1....
-".
.......4.,:
,...0.0 ...
44 v.** **NI. *4 .......*
VP, N
M4
,; 40
ise.... yore, 1. .. Ye.
y ...e.
...........
......,V
....O.
/ftA
n a..i
.11.,.4
rip,....................*.. a . .... **w
e*. a**,
..... ,.....60.
3es
..1.
....
ekS.
40./...
-40-
taught -.4d Column B Indicates the percent correct on t e
objectives that had been taught.' The information is:
cumulative for each test period for the entire .teMester.
Teacher Summary Report (see -fig. 12)
AiThis report lists all the students in a teacher's clasS
with a scOre:on thetotal testa
Content Summary Report for School Rrtncip61- (iee
This report is organized in the same pattern_as the
teacher's. content summaryreport. HoWeveri the Principal.
report tells how all the students at a given level or grade
have performed on each objeCtive,for the level or grad
Besides giving the principal a feeling of-achievement or
lack of achievement at each level, this report provides
him with some specific information such as:.
a. ,How many, students,have been taught for a particular
,z)bjettive at each level. .4
b. How well groups of sOdentt'o:e-performing on those_
objectivet which have been taught.
In addition,,this kin&of report can be used by teachers
at a.gride level for re-grouping students for review or,,
introduction of malerials according to similar deficiencies
or accomplishments.' Altogether the informotton on this
report:helps provide abotit for discussion obout student.
growth inon academic subject.
4n.
=1.11.m
.
TREND
T E
11
kci4-ER
TEACHER NOI 626
--
SECTION Nti
01
LEVEL
-
i06
TEST PERIOD 'NUMBER
-
STUDENT
NAMES _
i2,
,,
3.
.
.'A
0c
D.
A8
c0
- A
0C.
0''.%
'0
C-0
MilAvOLA
STEVEN.
25
60 11
82
25
76. 12
82
P3 .64
888
25
64 14
79
CORDovA
PrTER
-
.25
20
2.0
25. 28
20
25
12 ,
10
25
20.
10
;.-CUNHA
ARMANDO .
-25. 24
25
28"
'
DEL VECCHIO
.)I09FRT
---,25
20
5 . AO
25
32
425
P3
32
g :so .,
'25
24
250
-
FERNANDES
.SrVERO
25
4,41
9.
67 ' 25 , 68
10
xn
'215
68 12
92
25
72 13
85
FUSCO
-NAYM.
-
25
56
888
25
68
8 100
Ps
72 '8
88
25 -80 14
81
OELOSI
JOSEPH
25
84 11
82
25
92 12 100
23
9.2
388
- 25
84 14
93
KALOMERES
OFOR3E
25
28 5
40
25
16
4,
09.5
24
050
25. 40- 24
50
.
25
20.. 1 100
:MALDONADO
:ELIZABETH
093
25
20
10
25
32
1P
24
MARROW
.
-BRENDA
25
20
i0
25
20
PS
36
1 100
'2;
16
10
:.MARTELLO
JOHN
25. 20. A
50
23'. 28
5-
20 . 25
40 ,4
75
MERCADO
RICHARD
25
24
pn '36
25
16
.....
.
NOWERA
RICARDO
25 28
540
25
20
250
25 '20
450
DLIVENCIA%
RAyNON0
25 24
25
16
10
at .
32
10
25
40
1: 00
pATERNO
'LAURA:
25
20
2:0
25 ''20
20 E
pt. 16
1.D
25
36
1,
0
--
.
PROUT -
....
MARK
25
36 '1
025
24
10
25
16
i 100
25
81
-,
- REUTFR
-MiCRAEL
25
40 ,1 100'
25
40
23
36
10
25 24
333
1 ,ROBERTs
.i.,....;
ANTONIO
P5..28
425
- 25 ' 24 4
50
PS
32
560
25
48
863
F1.054
_:,
MILDRED
'25
72 12 92
25
64
8 100
at
76
8 100
25 64 15
80
ISMITH
JAMES
25
28.2
67
''-'25
16
520
25
20
729
25
40
8- 25.
SMITH
INEZ
,
.-25
32
:1
, 0
25
20
10
PS
28
10
'-' TICKLE ---
CDLLEEN
--.725
24
-1
025 - 36
1.
0an
20... i
..0
25
36
1 100
kAGNER
ROBERT JR
25
40 "-3
50
25
60
978
03
60
978
25L 76 16
94
7- 14AL.SM'
'
TERENCE
25
20
i. 0
25
32 i ,100
PS
'44
'.1 100
-
25' 48
580
---'111ONO
bielE
7 25
56
475'
25i
-
48
875
25
52 10
90
.
25
68 15
73
f.
......
ATOTAL RESPONSE8
AttPERcENT CORRECTOP- TOTAL RESPONSES
1A)
C r 70711.i$Or
POSTINSTRUCTIONALRESPONSES
O la PERCENTCORRECT OF
P08TIMSTRUCTIONAL'RESPONsE8 44
MI MATH
SCHO
NO1 0445*
5A
8C
0A
25 60 2i
62
25
12
30*
25- .28
4.
75
25
32
5 - 60
25- '76,-13
77'
25, 64 21
71
25
84 21
86
25
20 '5
20
25
20
30
25
so
30
25 '32
580-
25
12
2-0
25
AO .
782
25.36 4
50-
25
16
367
025.244
0
25
28
40
25
28
633
25
88 23
91
25
40 .
771.
25
28
30
25 36 4 75
25
64,22
64
f25
60 20
65-
r.
11.
,..444,44-44-4,44-44,44,41444'-7777
.1:7)
TR
EN
D'T
ES
T I:
CO
NT
EN
T7S
UM
t4AR
.RE
pOR
T - F
O R
-SC
HO
OL.' P
-RIN
CIpkt:
DA
TE
: t
SO
MA
. NU
MB
ER
a.9.44
4S
UB
JEC
T: M
AT
H
-ST
UO
EN
T-O
RO
UP
, LEV
EL 42-'
..
------"--k?
TE
ST
PE
R IoD
NU
MB
ER
: "-i.
23
-4
OB
JEC
TIV
ES
AB
CIX
A13.
CD
AC
.A
C
14201004
\)1135
'42 -57 '-"/4248-70'. 56
14662
97
1-420006"
0117 ---34
24 ---115 23.---5V 36'
119 -39 -'97 -.4314201007
2153
8358
114' 5570
9352
- -14201005
10856
5364 -1119
59- 71
5,6=-'
12073."'1.01
73
1420004 -/420200514202006142020071420300114203002
'-142030031420300414203005
*
1.4203006r,,142030071420500114205002
~.
5B
C
.sC
14205003etii420 60014-14206004
1420600514206006
.1420600814206009142060101420700314207004142070051420700614208002
- 14208003ABC
s. 11248
60'48_,
11454
,50
52111
5973
64
10173
8479-
11577
9778
19 72.97
7.4.
10866
4-- 176
119 *66"-82--..72-- 120'
83 -70-
14631
-20
30152.
47100 -51-- 150
_.83...'12352.
- -113
3454.
(39125
4059
46- -108- 45
5551:
7'-110
"-" 56- 129
-47.77.76-'55---66.-16 -71*
122. 2025
52*114
:2521
48"-101
3755'
49
12068
64- 94
12961
70109
8799
96
1.12az
35 -94"
11464 .*:;32
81--111.-88--' 92
93" -
-** -
11930
10 --- 129 -
1- " 0:12k- 35 --21 ."24
0
.
0
11072
50-66 -115 -72-.7571-7t -124' -69 --:73*-70113
6337
.57125
7939
69108
8374
8454
,63 _112
7939
8264 ;.4 58-
64"55-60
78"
73-123 '76- 76
7875
5483
-z1244"79;'" Ai
78
42 .:i .16iss
109 -443T
- 4189
49is
. 111 .92.
5684
60.61
62109
609-0
6050
,55---82- 55
62 .: 22 ..687*- 10961-- 57 ;58
4869
- 11656
S8 -69
----*
,t 9-
62122
61_A
la75
11071
111.4/
112. 86120
53110
44120
53121
56
2070
118- 52 '
12557- 67
-125
5267 -
11517 -59-.11539
90114
2273
129:129
"..- 129
-4J,.
..
.. ,. 0 ..,-.
--...
,..
-9 0
'-
cn, (I) 14... 4-3,
".,,..
...... .-....,. .-......"..."--".....,-............... -,......
--......................... - ....1.... - ... -4... . . . .........4 .i.,..,
e" -0;.,-,.
..
.
9_, .-.0 tu
.1131
JO7 ......._...........
-.77_7.__ ...............
....................
......................... - - ...
......
.
___,._,.
............-................-....,...........
-......--- ------..r.------- - -----. - .---..-.......-
7",.
.
4$.6.: tf) . ;C.
.,
.
.
-'
VI 13 U
.-.
...
.".
C.4
.'
M It .
..
..1
III.
-I.!I)
.-
.
C U
l..
ra
11051
-- -
11565 :;'``
71f.
12224
12252
11456
;569
10152
4957
0-0
114 -61.101- 57
11991
11885
..
--112 -.87 --4k--
.44
-
.4
a TO
TA
L RE
SP
ON
SE
S.
PE
RC
EN
T C
OR
RE
CT
OF
(TO
TA
L. RE
SP
ON
SE
S ( A
1,
TO
TA
L OF
PO
ST
rNS
TR
UC
TZ
ON
AL-R
ES
PO
NS
gSII PE
RC
EN
T C
OR
RE
CT
9F:- PosiNv
sTR
ucoNA
L. R
ESPO
NSE
S rc 1-----i
amilm
*.41.4.4.444.44404
.
toest. 14
OP 14,14.11,1440440.14
44.4444.4.1.44 44... ,....
.0.44.4
'
--a.
4. a 9.41.1,
.a;
4.44.=4...
JR.
**.'
.19
4;37,
* Content Su ary Report for Administration. (see fig. 14),
Ibis report\istimilar to the Principal's report. However,
it'providei information districtwide,about student per-,
formance at rade levels:on different levels. This in7
formation IS invaluable In considering qUestions of\i
a. effeCtiveness Of-specific materials.
b. effetttveness ofAnOgrems,,methods
c. appropriat nesS Of partiCular.objectkieS.or-test Item*,
91. of materials and'probiems.
. +Curriculum plant erstOo oftenp jUstmakeudgements about
prograMs without\informatiOn systems. 'Without such a
'system there is Considerably less rational basis.
.Student PerformanCe*Analysis by Class. (see fig. 10. .
This report which is provided to each teacher after each
test, twelve, times a year is very important to most of our
teachers. It-lists all the students in a teacher's class.'
AtrosS:ib top of the pageeath objects e for the:level Is
printed. o the right of,eaCh students name there appears
-the dest n tion
C = Co r t.for that objective4
W g for that objective
No e ponce for that objective
m ants .correct and taught and incorrect and taught
At a; glen teacher sees individual perfo mance objectie
0
isieCT
zvEs
142010:414e014).161420101714-2020 ;814202:':41.42023;,5
-142020.4.6-1420200714203:3
1.
142C3C
C2
142030 E..3
1420304.
97014203G
55. 14203036'14203: .7:420533114205:02
978.1'4205;-,3
985-142060.1.-1
96814206,i7;4
99'314206C
05 -981
142060 .5969
112206V.:8
913.142365-9
. 970-14206010
992-142070,.3
970142C
7=4:4
983'14207:35
9811,4207%
6971
<14205032
9711420801
985
TR
EN
EC
IES
CO
NT
EN
T S
UM
MA
RY
RE
PO
RT
FD
R,A
Dm
INI81R
AT
JON
SU
BJE
CT
: mA
T#
2
A- B
C.
DA
B.
D
'1227 -40163
47i1295
41260
k7991 -
- 12334
1040 'tab41
98330
13745
104030
20751
9e352- 279
61-1015 54
352 .59993
37229
461028
42254 -,51
991.62
48867 11339
64706- ".".:67
483: 49303
631014
55489. 72
1226 .2980.46 '1297
31191
44.
97833
2&8'
411027
'35277 48
99241
21860
100545
29261.
97122
9743
10252$
93°47
51247
25.1002
st2,86.
76
99360
208d6
102860
230113
98234
3946'
/01632 '
*6939
981 16196
4'8105d
39190
60
98.161
28071
105162
41364
63.283
691027
68'374
7251
128 .81 .1003S
i121
70
59- 19967
103860
33259
70287
74- 1042
68 ,44474
61.151 .68
105063
29971
3574
5014236
37132
50
82
1,3383
1028' 81216
84
50147
731002
55275
67
4557
46tops
.45117
40
52-19
581002
5153
47
4519
53.1040
4553
53
4916
501051
Si
4134
3117
471025
3395
57
5117
761025
5412
'67
Si
-39 821003
79106
77. S
TU
DE
NT
GR
OU
P A
CR
OS
SS
CH
OO
LS-LE
VE
L 42
TE
ST
PE
RIO
D. N
UM
BE
R:
3A
5C
D
1275 .49402
541020
29,` \ask:1029
34t.30
511028* 57
47866
104345
42555
102062.
71967
-102859
624.69
125737
29951
102336
36347
103449
372U
r;4
eIre448
91400
88142
35293
60443
71482
74240 ° 85331
6447' 78361
.74187
51280
86401
64235
54124. 56122
64°
9865
10851
6475.-
28086
A
-
1C25.'
104370
101032
996'46
99765
102370
1051 .63999
6210.01
996 .7'..671017 -
381043
811.029
571014 .491029
52.1,020
52'.997
.58
104033..
104058-
'1051.84
A4T
OT
RE
PO
tiSis
B ta P
ER
CE
NT
CO
RR
EC
TO
F T
OT
AL -
RESPONSES.
(A)
CZ
OT
AL O
P-P
OS
IINS
TR
UC
TI.O
NA
LR
ES
PO
NS
ES
0 it .PE
RC
EN
T. C
OR
RE
CT
OF
pOS
TIN
ST
RgC
TIO
NA
L.,RE
SpO
NS
ES
+
c .vi ,.
-V
V...,
vr=. _1N
.
"+""
034ru 4-4.I 0 ...ti)
,>.' 0
1 >C
. :44 0...
V.) ....
E (t)
4 ..c.):411
.:::(19°iljuM
i
1 : it
;1 :
....72 1- E7:
9
-4:1 : mou°
0-001
-0 - ty 44
'o I- t--.
C
II) 4- .0,b, if:
I-t- .c
`0.0'61) 0
E. In0
4/41.4.1--ti44' .0-C
° >. cn
-z 4a
tr, 4... cC
VI
<I)
....-13 '
CI1...
CI =
C -I
4JC0 41.
r" 'U
TE
AC
HE
R 401 027
.tfilrkSE
NO
r 000'
*SE
CT
ION
NO
!01
ST
UD
EM
T404tH
i.
.
DEBRA
10r/01..0.....74....
c410i"=
"Z
.acar7
i.
s1). DENf
1 1 1 1 1 1 1 1
5,5 5
55 5
11 1-4 1 I
1.
- 1.1 1 2 2 2 2
0,,0 0 0 0 0 0 0
* 0.0 0 0 0 0 0 0
1 3'4.12 3 4,1
CW W CV
. RU
SS
ELL
C C
.'C C
TT
'T
WILU
AM
WW
C'C
.T T
T T
DE
BjkA
'W W
C C
ITT
?W
ILLIAM
RIC
HA
RD
RICHARD -
,,,RO;n0WAR4
;-
RA
PH
AE
L
MIC
HA
ELs
;
KAROLYN
PER.FORWARC 1.
AN.AL'YStS
0J 'E CT
IvtI
11° I
I .1
1'1 4.
s
5,,5 5 5 5 5 5 5-'5, 5 5 8 5
1-1 1,1,1 1 1.1 1 t,t 1
S.
0 0
-'O,0. D
itf0,0',0 0.0 0 0
5.5'5 5_.5.6 6 6,6
4,-66. t
0 0 _0 0 0 0 0 0".C
at 0 0.00,40 0 0 t1.-0.0 0 0 0 0 0,02.3 4.5.6
3 47 8 9
s
CW
staweccw
14
T' T
T
W C
*T
NCCCC
C N
CN
TT
,
ai
S507001T
8.:4YC
L 'A
9-
'WC
002-15
1 /5 5
1 407 7d. 0
11
1
'5 5 51 1
1
0'0 07 7 70 0 00 0 06
7 8
//
5 51008800'0 01 2
CC
1414
itit
1'TC
C W
SUB
JEC
TI
'MA
TH
,S
CH
3OL N
O1 12
rut No.
2
5 S1
11
0 0 08
80 0 00 0 05 6 7 5
,.<
(0..>4-I
13....I
C"" 0
w.CWWC
wC
CW
;
CW
C C
WC
W C
C,
CW
C
_-
CC
C'C
CW
WC
CC
C C
CT
.I 'T
4
C-W
WC
W
7
C W
CCC
CC
CW
'CW
,T
T4
'w c w
twW
K.
C W
WW
W C
WW
W-C
"C
W W
,TT
'.
Is4
Wc
CW
C 11 C
;14,W
WW
WW
WW
WW
CC
T T
TT
T
W C
C W
wW
CW
WW
WW
WW
T- T
T
14; CC
Cw
WT
TT
T
CC. C
:C C
T T
I T
CH
HC
CC
T T
T 'T
C W
CW
CW
w,C
W.:14
t T
C'
CC
CC
-14CC
.WC
.T
T
W_W
.CC
CC
CC
CC
wW
WW
W
wc
ww
w
C C
1(C
C C
CW
C
T T
T-
setom KEY $ CsCORRECTo 1001110NC, NotiOs RESPONSE,
T*POSTINSTRUCTIONAL
CC
CW
-C C
WI
T
C'C
CC
WC
C.
T
CC
C C
T'
.
C.
0
I
.
by Objectivt.and the potential for grouping accordingtor.
SiMilarittes.and differenoe5,1nperformanCe:for the YaPious
objectives.. Deborah, Richard, 'Ray and Karolyn would seem
to:be:a natural study group for the review of objectime
`45101003. Eathof theSe students scored a wrong answer
. though this objective-had been'taught. Many other,such
"natural" groups and CoMbinatiOns are readily apparent...:
This report is a good eXample of teacher cohtribution_
o the improvement of,a program.. All this information was
'aVailable before this report was developed.: However,
teachers repeatedly requested.thiS format because of the
visual advantage it provided. They insisted that looking
at class performance '!at a:glance wOUld'make all the An-o
-forMation:more usable. When the data,processidg center.
finally succeeded in producir the. report there was,a
quick acceptance 'Offthe report and use of the information.
.-- Also after-each test, each school is .provided with a
Student. Bank Update. (see f ig. 16)
.
This is a listing,of studgot% in the program after the
changes have been made for this period. Team leaders
(explained irk section on implementation) review the list
for make sure all students arecOi-rectly entered.
Sfinilar print-outs are available on request,from the0
bahk of objectives and test items:
D' ' 0 ' 0 0 40 P 0 0 0 .47 P o r2 0n 1
... --a .4 .4 .4 NI .4 *4 .4 .4 0 C. 1. cr : c. o os... I .. . 0 , r, .r so to sr 11 1 rc . Or VC
4/ %.I .4 c o in ... .4 .4 W Ce .4 1 CO -IX 4.1
0 1.0 0 111' CO . CC CC gra .1) .4 0.1 c.1 1 cc... %a js,s
I to .... (.1. . 141 i 0 o .4 f. trl i in - N at 0 0 'a'1 !
t
s I .1
1
To Co = w co : t22 U) ID co cr CO AX UX 13 V 0 V , 21 X/ 27 77 Z V 0 .4,, 1
, co 2. I. 2. 2. I 2. I. 2. 3. 1. I. 1"- I.. 0 C' C C: C": CI ... 7 7 0 0 o ' 0 1, T7 to RI Z 2. CI 0 t7 0 k 1 C ..< ,,,C
:,,r or .73 x 0 o CD
74' X X ,
1
in 4,,,
4.3
CU
C0 0 0
01.0 71 CP CD U) 0' c . cm dm .'N r
t13 w, r CI o 0 0 o c2 .0 2.... , .2- -C -c -C -C -C -C -C-4-2 r2 r- r .0 CD 0 0 0 0 -1 .PT m 1. FT CO rn
0 0 e 0 s.z
1.
I , - I . 63
1 31,., .x. C7
CI 0 r 11- CO Cs
I T-. rvz cs
..»m
I IT D. CP /0= XI
00 co
fr. :01
0. 6+. 01erQS O.r
!OA
r i 1.V. 9. *.^. .'- 4. N
CA 0.I0 0 40 0 0'Ca
/8...1.1.11 C7
0 A t)T.1 . N ro r
P. P. p.rij n)
trt tr. 0P. b.'0 .0 0 0 0tA> 47 NAA,
0 UP.0- F.Q 0 9: Cy 4, 1.1
.. .. ,0 0 0r
0 0 0 0 00 0 0 0C 0 0 00 0 0 0 00 0 0 p. 0I
131.0 0
0 0:00o0.0-000 00 CI0000 007 00 CMto r
0 01
0 CTW
0to
04/1
rC5-4
0
N
O
r.
U1 U1 ,-1CU V IV 1.1k 0 0 -
417 0' 10 0 0'0 _00.
0"'' 0r r r000'0 " OP. 1>
0 0 00 0 0.0
ca 0
0 '0
... .. .. " ,.. .....c, 01 F IT r 10 1+ 471 FOr . r ...r .- r.ocz , 4- a. ...' cs. -- CIr ...
0 , .4. 111. lab 3.11 . Fcc .... ..a .... + ... ....
0 Ul. 4 P P.'6 .. j,c c r. r : .... 6.: p+ Ii. ..
C U : 4 , r 1) 4-CO or - .r,
r w ..5+ rt) 1- to - ..4-, I-- 0 .... lit 4.-6... IF. F. )- F.. F. r0 0 0;o 00. 0-41" Ul 4, fr 41 C11 CU
0 - r. la 4 La - - -10 - r 1- N ) CO, - r0 0' 4- 0 - c P. N' 4.c.+ ..., 5,.., .,. ,.., ..,,,....,. .....,
0 C7 0 la C7 CD c) a, .C710 . 4, 0 . . Ln r 0.. 63
1 I 1.... 0+ r r r r r 11+ s... r;%3 ,,... .... ...+ N r to i. r .. ....Ott r c. -- st. r us r .r - in
0 0 )0rO 0 0r0 0O 0 0ro 0 o 0oO o00 00 00 00 00 00 0r <7'1`.1 C0 0co oO 0, r O _
.0 0O. . CD -0' U 1.710 C7 -O 0 CP0 0 r
1-
7rn
0.e.
:C
L
O
4 ViOD Cu NCO W. Ch
0 1:7 00 0 00 p. o
N
40,
'1
-4Lt 0 0 a. in. in 0 -7 CIPP , AI 11.1 Ci O 0 110 4O r 11) 11.1
1
44- 11. r CD CC rO 0. a a 00 E a 00 n ., ,..0 0 0 0 0 0 0 0 O. 0 CD 0 0 21 4
0 10 CD O 0 0 *4 0 0 0 0 ,. 0
4- [ U1
.
r 1+1
4- LI UIO.. F.F.P..
o- I r COI#
C)F.- 4- -
.'r 131 UI a r C.Ira r - a - r0 0 0 0 00.. IS. 141 0 11 01
r O 1. -- r 0 IV -4. to us o r 0." - r 0 V. V.0 0 0 0 0 0IV 0 4, 0 co ...,
[
CP
3* CA 0 4-tar1. 471 0 F
0r.
O GP' 0 CIC OF0. *+ P. P. P. 0 P..3% 13%
I..1~
r to
471 ?. 0 0 'r, tar
0 0
0 11-- CD CU -UI 0 ..'.13% 1).
4.12 F. P. F. F. -F. V. C) D.- P.0 0 0 O oc. 00 o o 'c o 0 0.0 o
N Co 0 rAA.0F.' 0. AA,
Ul 0: -r*A
I0. .0
0 tr. 0 r 0 -0 CO 4' 0. 0 0' 0 VI" 4' 01Or . r. F. 0 IP* OrCD C7 O. C7 C7 O CD C) C7 C7C70 P DJ C7 0+ C7 n3 UI 4
1 1C7 . C7 ea r r . C7. L. Or . o+0 r 0 0* r r C7 - C7 r r0.01 ,0.t.n 40 CP.. C7 0 C2 40 400r 0r C) 0r V. P.00 00 0 0 00 0U 0 00 h) 0'r r 0 1.11 0 ...I C71
0 p o o 0 0 . -0 o. O o0 0. o 0 0 0 0 0 -0. .0 -0 0 00 0 0 117 0 0 0 0 0 0 0 0 0 CI 0. 0 00 .0 0 0 0 0 O 0- 00 0 0 0- 0, 1.7
O i O 11.1 0 0 0 o C. 0 0 0. C) 0 C) 0 00 0 0 0 0 0 0 0. 0 0 0 Co 0 0 VD 0
I IIfig. 16)1
41-
r. -0 -0+0
- o O rC>
UI 0 0 0.0 CI - to0 0 0 0 0 0C. CI O a to
C)O
U)- UI c> O ae. 0 O ..0 0 0 0 CDF. .0: 00 0 0 0 0 0
OOC3
CD
n3 V. r0 NrO 4- 0 4"to 0 CO
C) O 1.2 0- 1U 01
D.. P. V.N r%)0 0 Ftar O W00 0 o o
ra W ror 0 Q -0r 0 0
F 0 U4Or 0 0r00 0 00O o n3
0 1- 0 0 rOr - ++OF 0 0 4-0 r 0 0o . U c>0 4" t).-
0 0 0 0 CD0 0 0 0 000 0 000 0 0.. C.7 000 0 0 000 0 0 tO
CComputer '6rint-put of current student bank0
A"
7 0 7 0 0 P1
.4 0r0 o p x. na
Is Ca
E r11.
6. to #t."CI. 4" rim m
r-r1-.. P
.
r -4its -- tu asr a- .r p UI 1.1)
00 0 o01 -4
CV - r CU r m.- 0 4' r Ln4.O C> 0 0 pN no Ut r1...n) r r. 11.1 - C')- C. s r 01 in
-110 0 0 p 0'0 43 C. IV 0 CO1 - - -0 - 4-0 c. OU1 us,'0r -- pr +410 0 0 0 oo r r 041 SO -- o r .4r r Or0 C. 4- 0 111 1i)0r p- Ur -s00' 1.) 00
:1.1 0 .4".
Alt 0 0 0 0 -41
0 c...y U 0 0 PI0 0 0 0 0 00 0 0 0 0 -40 0 0 0 00 CD 0 0 O 0.
G1
-148-
. Item Analysis. (see fig.. 17)
AL the end of each semeste; the coMptiter program auto-
Matically supplies to the central Office an Item Analysis'.
This report lists all the test item nUmberS down the left
co'umn. AcTost'vthe top ere listed the choices that wereo
available to the'ttudents. .110 response - meaning the
student chose not to °answer question. A B,C0D E choice's0 en .
both pre-instructional and'post-instructional. Next to
each .test item number there is indicated the-humber.of_
times a student made.one of the twelve possible choices.
This repo is extremely mportant-in validating the
test items and improving the, quality of distractors. In
the middle of the page test item 2080500102, twelve
Students thotganswer A pre7inStructionaIly and Il students
chose answer D pre instructionally. This'is usually a
.
good inditatiOn'that there.is a problem with the distractor-
or the right answer. It is a gOod signal that this item
requiret re-writing. Similar patterns-can reveal problems
with-test items. Constant refinement of the test items
will result ,.eventually, in a set of test items which are
tailor-made and standardized for the local district.
asc
z0
,-
,...1
1 1'
00
2*
34.L
0.=
k-t.
.00
06
0-
0.4
..
T7
0:
00
4.n.
40.0
Wow
....is
m..
00
00
e 111'
n
'.0
'
ti11
10
T
....
0.
0
74.
00
oi0 0.
r0
0.4
a.m
.. m
r.
...
..
0 0
vim
*
c_:*
a-
0'C
0
04I
0 0.
0i
.T
4.-
0G
T
0o.
e 9;0
.-4
.9t
0 0
eo
o.4
10
ft0
.0Z
00
0OZ
06
0T
0S
0
0'II
09
.d
0ir
0;I
0
.0
8.0
.q
06
0
..!
`0.
',"
0'
COV99fiVic
0110i
No
totO
cnol
:_.
.... E
,0
1
777
201003080
--
.
1010
0201
60
.2.0
0100
t02:
.T
t...
30t0
0/03
0
20
tOt0
0z02
:...
am.
...
oo
4'0
Z.
0 0 -
0
S0
-'4,
2'
0
co.s
oolo
st;
-za T0060915
T01009220,
4tT00202.1
Z0:
0090
20
Tot
ocis
caz
0S
T0
4b
90
t 7-7
7 0-
74ot0O*;q0
0-
0t
0,
i. Z
0-
Si
0-4
.
taigotait
-0
.0
0t
0Ea
0t
0Z
0'
a?o o aloe;
swam
is.
06
04
0IT
0L-
0TOTO 04020
00
I0
E4
40
elo
90
t"
co I obe ova
10
-0
06 e
002
.M
N.
MN
mi o
w--
.
00
0...
....
.27
..
.
0a
o =
0.
-zoloccogo
Am
"
o,
o6
tlo
tooe
cso
-
.19 0 (17 3 V4I
O. el
31t
oil
3s
47"-
-1s
313
..a
0.,
2Y
S30I01-13
.
ON
I`
W311
1. w
SL/0 / }Z la SY ON 20y3
u04 SXSOYNY 14312
ve .
.
_Computer Procedure
1. Brentwood has an NCR Centuri, 200. The computer has
2.
32K memory, three 657 high stack disc drives. Companidha
equipment includes a high speed printer,. card reader and
Op-'Scan 100 mark reader.
All banks of InformatiOn are originated in the curriculuM
office which: optionally tubmitd test items coded on the
data proCessing forms,
. All forms are keypUnched and cardliptings are sent back
to the curriculum office for refinement or confitmation..
Any corrections are rekeypundhed and a new cardlitting
is generated. This process continues until the card
listing is confirmed correct.
. Objectives (and any items) are then stored in computer
accessible banks. The LIST.routine generatet a district
0
copy of the bank. As curriculums are refiaed, objectives
can be added,-deleted dr,changed,in the cOMputer banks by
using the ADD, DELETE or CHANGE functidn respectively.
',The curriculum office submits the appropriate function,
forms, and the process cycles.as in step.-3 until accuracy
confirmation is received.
Curriculum Office submits COntinUoUs Trend Test-construction
forms. Process cycles as in itep 3 until accuracy con-
firmation is received. :
-51-
The computer generates Continuous Trend tests for the ,
. .
semester. :These are sent to the district for diStri-
button.
'Answer keystfor the semester are. then TUt up in the com-
.
put banks. They are-either automatically generated
from optional master answer key files, or elSe"the cur,
'riculum ofTlce must submit AhsWer Key Forms. in the second._,
easei Alle...prOcess cycles as in Step 3 until accuracy con--
,firmation is received. In either cate answer keys are
est blfshed and a listing is automatically generated for. ,.
the ld ittriOt s use.,
A student, data base it established on the computer either.
from an:wasting base or fromCinputforms.' If the base is
established from inPutJor01s, ffievprocest cycles as in,
step 3. In'elther case whekthe student data bese i es
tablished the LIST program will generate a distritt_copy.
After the student data base is established, the computer
generates a semester's set of Test Scheaulet'for'each.
-- teacher. These are sent td-the dIStrict for distri4ution
and Pverificatior z4
,
VO. When Tett Schedules have beeh verified, each teacher re-.
ceiyes a semester's set of preprinted-student response
sheets.
. Teachers submit any student file function requests a week:
60
before eaclitest.
Forms -are keypunOhedand a card listing is .made for
verification.- Any CorreCtions:ere rekeypunched and a new
listing is Thltprooess"continties until thi-Aistipg
is confirmed correct.
13: Stu nt file maintenance IS then procesadd..- ThefLIST
programgeneratea.a risiing of the refined student date
base. This is sent back for disteibutiOn..,
14. Teachers administer test's.
15. Marksense forms are converted to Op-$can cardS.
sample of .5 oUtof each 100 cards will be.checiOd to
guard against hardware failures,on the 0o-Scan machine.
6. Tests are processed and student/teacher/content:reports
are generated for !distribution (15 hour turn-round
During the summer of 1972 Public Systems Researcli,provided
consultation services for programming the Brentwood computer. The
program was in Fortran IV and adequately served 2000 students. in
4th, 5th and 6th grade mathematics. 'However, the cyrriculum office,
reading consultants and teachers had already selected objectives
and test items to begin Reading Comprehensive Achievement Monitoring..
Difficulties with Fortran IV in the NCR Century 200 precluded
anyexpansion. In thg first half of 1973 the Coordinator of Data kk,
Processing, Mr. Joseph Rotolo, undertook to reprogram CAM in COBOL..
The resbits surpassed expectations in the following way:
6
-53-
. turn-around time reduced
capability expanded to handle .up to four progums for
22000 students
. increased. Accuracy
We now process tests at 9e (vs 25t at Board ofCooperative
Educattonal-ServiceS) per student: This is basically an expenditure
for paper since all equipment and all salaries were being paid for
'business and clAricalfUnctions..- The fact that we now process
"snore than 500,0 answer sheets very.,test period without adding 'to
costs indicates hat personnel and equipment were previously under
utilized.
Vp
62
z?
-54'
DT, Support Materlets DevelOped for the AbnitOrIng Program.
The most obvious need for suppot materials was fora publi-
cation that Would. open the instructional program' for everyope in
the school district. A committee of administratoes and teachers
was tPrmed early in the program to produCe such a booklet. The
B.F-.S.T. Manual for St4dents and Parents (see appendix .8) was the
outcome, The booklet attempted the following,:
1. To explain the instructional program before the Stildent .
. receives:an evaluation.
To,p44-ain that the'student Wilr,be tested regularty and
that reports will be sent home
`to list the Minimal set of objeCtives in mathematics for
eacir level.
To supply sample test items
a. ,to give an indication on how the students knowledge
and skill would be tested
b. tO.help make the objective clear by providing an ex-
.
ample:
This publication was distributed to students and.paeents at
,the beginnihg of the schooi year. It was generaltif.well-acceptea
except for a,few parents who thought that the instructional program
.
was` Oeschool's'business and should.rematn.that way. Many parents
did not react. .Those who did usually had praise for the booklet
and the communication about the.i'nstruCtional -.program.1,k
For'lurther suppOrt of the monitoring program, the BEST-;,-
.Committee decided that it was necessary to examine the materials
available to Brentwood teachers and to cross-reference the math
ObjeCtivesto those materials. (see appendix El A CrosS Reference.
Guide was developed for each grade 4, 5 an.-,
The adopted math bOoks were the Houghton Mifflin Series, We
.
had on handy some new texts.: some .old texts (which were left-over
from before the Houghton Mifflin reV4t0n),*and thekddism. Wesley
Alternative set of texts (made available to sChoolandsteachers, ,
reluctant to adopt the Hotigton-Miffin:), .These teXtS and the0 ,
audio visual materials which..Supported.,the Hbughtorvlifflin Series,
constituted nearly the total available materials for teaching math
In-Brentwood., When the objectives:hadbeerrpu41-tshed in the Manual
fOr'Students and Parents, it seemed necessary. tO,Indicate where.-.
these objectives were taught; where cOuid'be found drill and practice
to help sharpen student skilli.
,
Level by level the objectives were matched to the materials
see fig. 18). This booklet now made easy a new approach in which
teachers could .now start Withan objective instead. of following..
text book sequence. Anticipating that teachers. in, reviewing the
.
tests would -request a hiridy and usable answer-keyi it was decided.
to make a booklet which provided each teacher with a complete set
of answers for each test item in the 'program.: (see fig. 19, appen-.
dix F.) The objective `is listed in the column. To the right of
TEACHER NOTES
OBJEC-
TVEt
HOUGHTON=
2-117.FLIN
N` 1st.. ED,
HOUGHTON-
MIFFL
IN-2nd* ED.
-,CROSS REFERENCE'GUIDE
tCDDISON=
WESLEY
2nd. ED
HOUGHTON-
MIFFLIN
ICSP
HOUGHTON-
TAPES
FOR FOURTH GRADE
HOUGHTON-
'
SRA
WY=
DRILL
FILM LOOM'.
TAPES
EDL
1. MATH.
-
_MASTERY.
242-21016,17;20=
23,174-
Ll75
4130
174-175
320
2-20
1-12
I-3.to
1-5
118
Unit I
3, 4
2-20'
327
'
156-157
'Unit I
1, 2
Knit *I
41-50
4160
194 -195,.
196-197
8) 9.
304-308
1-16'
162,163
236'208,
3
214
paste
-
$kills
Fractions-
1-14
IV
1-26-
O E
C11:11
CO
W0
4--O>
Ce U
*3
VI
U44 0
O W
.0 *-1
0
,kD'LA
N4170
198-202,
204 -205
1-64.166;
3b7
234,236,
237
1 -14'
.5,203,318.
4210
4p749,52'
246,2
317
7,
44-45,48
232-7
a
36-40
4220
46-47,50,
51,53
N
41
,42-43,-
46-
48
49-52.
Addition
I-1,
Skills1-3
8,9-'
SuOtrao-
1.2
tion Skill 14
8-
#2
4230'
118,121,
224-26,
228-29t
192v194
222-223
170-184,
Chapter
31
5, 7
Whole Num-
ber
pati-pnotioh
Skills 3,
-4;5,1;8
Z30-236,
214.-216
202-20r
Wholt
250-252
230-232
Chaptcx
'slumber
Irt5
BEST
BRENTWOOD EDUCATION SYSTEM YOR TESTIITG.GRADE FOUR
. ANSWER KEY 74-71?2nd. SEMESTER
OBJECTIVE
41140
2
A1.04170)180424042'50, D A r.4260 D C
270 D C
4310 A. B
432(1 C D
4330 B C
'4340 A D
4350 B A4300 A C
.4370- .B A
'4 510 D , A520 ' .44. c
4530 D B .4610 A B
11640 13 04650 A 13
4660 D
46o , A D
.4690 A:. B
46100 C c4730 4 C D
4740 B 4B,i44750 A ,B"4760 A B
4820 C B
4830 C D
3
C A
C*BD.
BAB. dA A C
A B Bit
C D /- AB A C
D D A'C B BD C B
D A B
D C AC .0 B
C D B
C A D
B ) C D.B C A
A "C 'D
' B A D,D A 0D , C, A
it A D
B A C
A D ' D
de:. D A.D c C
B. 4 A. 0.A p . B.
31 objectives X teis = 121+ = 2 items*= 126 items 6 tests = 21 item test
Answer Key
7.
.
-58-
the objective i listed the corred't answer for each of the test
items developedfor that objective.
During the first year one of the. BEST. tearli leaders felt a
need for the development of, practice activities for each of the
objectives in the 6th grade curriculum. Working entirely on his
own, Mr. William Harris of, the Northwest Elementary School, wrote
.
his book of supplementary exercises. (secAppendix J) The Cur -
ticulum office was sufficiently impressed with this activity book
1
that, it. was PO nted 'and- distributed tO all 6ih g,
de- teacheTs.
a Visit to our: project a meMber:of the State Education Department:-
.
asked for some copies. Eventually /it was duplicated and distributed
-throughout the :State of New 'York,,
Also at the ecompletion of the first year, the committee of
teachers and admin-istratorS began to compile mastery tests (see
Appendix D) for each objective. It was a conscious, del iberaie
step in the direction'of mastery "teaching. For each level in our
instructionaleprogram a set of minimal objectives' was defined by
t achers and administrators. HOwever,even at the reading of the
0
list, it is clear that some objectives require .mastery 44in the 6th
grade. For example- #6210 - The students wit) select the number.
which is the correct answer to an addition problem consisting of
nd'more than four15-digit numbers. This objective should be ac-
cOiplished at a high mastery leVel by most fthraders. Normal/
6th,grade students. should get the correct answer nine out of ten.0.
67
_59_
times.
On the other hand, Objective #6130 -"Student will select a.,
base six number which is the same as a given two-digit base ten
number or will select,a,base ten number which is the same as a
given three-digit base six number." is the kind of math problem
and thinking that all students should be exposed to. We know that
this objective Will not be master& '(with retention) by all students.
To determine the level of mastery, tests consisting ormany
items for one objective were constructed.(see fig. 20) The uses a
ihe mastery test were then develpped:
1. to provide-information beyond the regular trend test.
2. to check or validate results on trend tests.
.3. to determine the regularity with which a student can suc-
Fessfully perform xhe action called for in a particular
behavioral objective.
In addition a variety of charts, graphs andiforms have been
developed by individual teachers. The BEST Objectives Chart (see
fig. li) was developed by one teacher and requested by many others
for making visible the Class record against the set of objectives.
This chart may be used in either the math or reading program. It
lists students and lists objectives. C = Correct - W = Wrong -;
=-No Response and T = Taught. Since all.the information of this
.chart is produced by the computer for the individual Analysis report,
it seems that there is some other benefit derived from displaying
the same results on a wall size chart.
ti
Et.E.s,.t,r, mAsrelRY ?EST
.04-7.03001
OBJECTIVE
TEACHER NAME
STUDENT NAME
.......................,......m....................
DATE . . ,
.
1010,0. 00001001000001/......
..
WHAT"-FRACTION: IS NOT 'EQUIVALENT TO 6/12?
(A) 1/2 (c) 316 (D) 12/24
. . T.FRAPTION IS NOT EQUIVALENT TO 4/12?o..
CA) 3/9 T . (B) 1/3' 219 '0) 6/18
WHAT FRACTION It NOT EQUIVALENT. TO 6/16?
.(A) 3/8 05., 21/56 (c) 22/32.' CD) .6418,
-
WRATIRACTION IS: NOT EQUIVALENT NO 10/20?
(AO 5/9 (6) 4/8 (c) 8 /16 0) 5/10
v
WHAT FRACTiON-IS-NOT.EQUIVALENT TO 6/8??
(A).-, 9/10.. 0)1 3/4 CC) i5/20 CO) 21/28
(f1g. 20) .z Mastery Test..
sample page
IIMEMMEMEMONOMMEMOMMOMMEMEMMOMMMERIMMOMMINIMMEMMEMMEMMOMMOMMEMOMOMMOMMMOOMMEMOMMEMMENOMMOMMINIMMOMMOMMOMOMMEMOMMOMMINIMMUMMEMOMMEMOMMIUMMOOMMIMMOMMEMOMMEMOMMOMOMMOMMMEMEMMIIMMEMMEMMEMMEIMMIUMMEMOMMOMMOMOMEMOMMOMMUMMOMMEMMEMOMOMMEMOMMUMMMIMMIMMONOMMEMOMMEMMOMMOMMOMMEMMEM
11011MMOMMEMMOMMO MEMMEMMOMMMEMMEMEMMMEMOMMEMMEMMOMMEMMEMEMOMMIIMEMEMMOMMEMMOMMEMOMMINIMMEMEMEMINOMMOMMOMMINIMMOMMMOMMEMMEMMEMEMMEMMEMMOMMEMOMMIEMMEMOMMEMMEMMUMMOOMOMMEMMOMMMEMMEMMEM NM= MMEMMEMEMMEMEMMEMEMMM M ME MOMEMIMMOMINIMMMOOMMOMMOMMEMMOMEMMEMEMOMMOMMOMMEMMEMEMMMOMMINIMMOMMEMOMMEMEMIUMEMEMMOMOMMOMMUMEMMOMMOIMMEMOROMMI__MINMOMMIUMME- MIMMOMMOMMEMOMMMOMMEMMOM MEMMIMMIMMOMMEMMOMMEMINIMMOMMEMOMMOMMEMOMOMMMMOMMIMMINIMMMMEMMEMMINIMMEMOMMEMMOMMEMMOMEMMEMMMEIMMUMMEMEMEMOMMOMMUMMEMOMMEMMMMEMOMMOMMEMMOMMEMMEMMEMMEMOMMEMOMMMOMMOMMOMMOMMMOMMOMMOMMOMMOMMOMMINUMMOMMOMMUMMOMMEMMOMMEMMOMMIUMMEMOMMOMMOMMEMOMMOMMOMMOMMIMMEMMOMIMMOMEMEMOMMOMMEMMEMMEMEMMEMOMMEMMO MMEMOMMOOMMIMMEMMEMMOMMOMMEMEMMEMIMMOMMEMMUMMOMEMMOMMOMMOMOMMOOMMOMMOMMOMMMOMM OMMEMMOMMEMMEMMOMMEMEMMOMMOMMEMEMEMMEMMEMMEMOMEMMOMEMORMIUMMEMEMMEMOMMEMEMOMMIUMMINIMMEMMOMMEMOMMOMMEMOMMEMMEMOMMEMMEMMOMMEMMOMMMOMOMMOMMOMMEMOMMEMMEMMEMMO MMOMMOMMMEIMMOMMOMMEMOMOMMOMEMWINIMMOMMEMMMOMMEMMEMOMMOMMOMMMOMEMIMMOMMOMMOMOMMINIMMOMMOMOMMOMOMMIMOMMEMOMMEMOMMOMMEMMEMMOMMOMMOMMIN
:t*
S4
V. How it Works. ,Once organized the CAM SysteM or the Brentwood.
yariation:B.B.S.T. provides the followIng:
curriculum defined by behayiorel:objectives 'Which are
systematically coded for identification, retrieval and
.grouping, and which are further classified by one or more
taxonomies.;
2. Test'items- or other behavioral indicators designed to
measure student performance on each behavioral objective
specified forthe system.
A set of-randomly interchangeable tests, organized so
that each test evaluates or a predetermined sample
of all 'the objectives in the curriculum.
4. Periodic Ting, usually weekly or biweekly, throughout
the period of thecourse. (bo-
-5 Computerized.anaiysis and reporting of results within a
few dayS of each testing.
6. Interpretation ofresult% bx_peachers.and students as a
basiS for decislons relating to curricula, instruction,
and estimating okstudent progress.
Modification of curricula,rinstructional activities and
the CAM design based on the results extending over the
,course.
1
-63-
8. Reporting of student progress.
In the Brentwood System,'the first test in the twelve test,
Sequence is used as'pretest. This system provides the added ad-
vantageof having all results stored in the coMputer so that the
very next test provides in- formation concerning growth, retention
and new learning.0
As the course progresses, the monitoring system serves to
provide feedba.ckdto student, teacher .and parent concerning progress.
It might schematically be presented as follows:
0U)00.
7C.0 01
C4-1 M
U04- 4-.2 0
m.000
Objectives
Curriculum. ctivitiesfor those objectives
BEST Tests to MonitorProgress, Retention
Computer Generated Reports
7-7-Instructional Module 3200. How poes,CAM Relate to CourseStructure, WillCiam Gorth and Robert O'Reily, SPPED.,41ewYork State Educafion DepartMent,U972.
te,
-64-4
The relationship of the monitoring system to the instructional
kogram is dynamic. The constant flow of information becometpart
of the curriculum. It provides a problem detection and self 'cor-
.rection mechanisM for the teacher and the course.
.7 3
-65-
VI. Using Koni- orihg Program Information for making Decisions
about the Instructional Program. Teachers, principals,-
students and curriculum directors have to make a long series of
.
educational decisions during the ,course of a school year. in
most cases these dedisions are built on intuition, past practice,
habit,.astrology, superstition and other less scientifically sound
methods. Asking the 'right kinds of questions becoMes'amajor
purSuit for those cUrTiculum directors, principals, etcho have
-
developed-an information system to monitorthe'studept'S progress.
Some regular questions are:
1. How .long does it take?
-2. How effective was past instruction?
3. .How do materials compere for acAieving-objectlyes?
4. How do teaching-methods compare for achievingabjectives?
5. Which sequence is most effective in achieving objectives?
6.. Which test, practices are most helpful in assessing progress
and retention?
Information from the monitoring system will provide feedback about:
1. .individuals
2. groups
3. methods-activities
4: materials
5. times
6. the testing system itself.:1
74
.4* -66-'
After schools decide what they should teach, a monitOring.
system serves ;several purposes.
a. It keeps track of how well students are learning and
remembering.
b. It provides information for specifying the curriculum.
c. It provides information for comparing effectiveness of
AlkhUs, materials, groupings etc. on student progress
toward achievement of goalS.
d. itiprovides information for faculty discussion of probigms
of instruction.. It makes team work more purposeful, ef-
ficient and productive.
e. It focuses the instructional rogram by regularly calling
to the minds of stUdents, parents and teachers the academica
[- behaviorsthat.are being sought and developed'.
information reinforces the professional nature of the teacher's
and the schools task: From the three years experience with BEST
n,Brentwood a teacher survey (See Fournier Midi, May 1975) clearly
indicates that teachers consider the information ,in choosing.the ,
materials, means and methods of Instruction. 'They frequently and .-
regularly report that while the systedmakds evalUation and assess-
ment of student progress edsier, it requires the teachers to make.
more and more decisions.,
A constant flow of decisions about individual students Must
be made: there must be an assessment of progress toward each ob-
7r 0
-67.
jective;' there must be determined priority.wWer for selecting
objectives for the i-ndividual to work on there must established
pattern of orgeniiing the class to arrange for opportunity to an
individual to work on his own problems.
Decisions about group progress are based on
a. average of .percent scores on total tests
b. group percent on each specific objective.
The general curriculum decisioh.about student progress,
whether examining indlvidual.or-group achievementusuaily s Made
in one or more of fbiai- Aatual alternatiVrs:
a.. The objective may have been too difficult, irrelevant and
should,be eliminated.
InforMation appears to be insufficient, calling for the
addition of objectives to be-taught and tested.
c. Timing is inappropriate. That,ls, that the Objective_
should be rescheduled to appear in some other part of,
the,sequence of'curriculum.
d. The tests themselves ase faulty and items should .be
'rewritten or replaced.
The information that provides. a basis for decition flow
naturally from a criterion referenced evaluation system. The
formation. produced by each C.R.T. should include the following
mihimpw
.a. - did the student learn the objectivet just taught?-
70,
. b. which Objectives will be taught later - which Shotild be
.pretested7 '(to determine lho needs instructions and who
doesn't.)
c. what does the student remember'from objactives taught
earlier in the course. That is, what is his retention?
Fronthis information a teacher and student can proceed to
examine the Reports. The student's report. (see fig. 10) gives in
graphic detail his performance on a set of objectives -that had been
specified... in the discussion with the teacher and examination of
his own the .student gains infotmation which
a. clarifies objectives - only working on them gives a,
nearly complete account of the beha0or'expected:
b. immediately reinforceS behavior which brought about success ,
for an objective. The teacherLsawarenesS of the behavior''
and the success or failure make it much easier to decide..
about rewarding, re-inforcing or helping the,student to
avoid specific behaviors; In many ways Aisactivi/y is
like correcting teacher -made tests. Howeier,sthereis a
great difference in the amount and form of reporting pre-_
. instructional, postinstructional and retention behavior.. 7.t
In- making decisions, for group or whole class tnstruction:
"How does one go abdut doing this? By methodically analyzing
and interpreting CAM feedback data.' The method used involves;
(1) detecting logical, indicators of progress or a lack of
0.7., `""'
-69-
progress, (2) 11Sting the possible causes, (3) .Asting the
conclusions (i,e., most probabrec
native solUtions and,(5) making a decision concerning what
must be done in order to benefit the student--thes
to help him to learn better (t.e. selecting 'the best alter-
native). This applies to "fast!' students as well as the
slower'oneS.
If anyone should 'ask yotrwhere,.in CAM feedback, one,PA.
should look for indlCators that all is or is not going we'll,.
you might answero'Evecywhere.'
'Everywhere', in this case, includes during,the.enti.re
*pre-instruction. phase (pretesting) immediaiety after in -.
..
-struction and In the entire post-instruction phase.
What are some of the things that can happen- during these
_phases?' Well, -We are a few. During the pre-instruction
phase you.may find that the student either already knew the
subject or/that he learned all or some,of it during that phase.
A high_Erttntimre,indicateS one or mOr#of thete posstbil-, .
ities.-.
A low score in the immediate postrinstruttion phase in-..,
.. '.
dicates that either some deficiency .exists I. the student's
learning or the efficiency of'the instruction or both. And,
a decreasing posttest score indicates diminishing retention._-.- .
,
, .
There are many other indicators.
-78
r
'74-
Aany or all of these situations came abOut is.part cif
the deteatlonanalysis--process. What to do about It is the
decision.'.
4
, -
5'' -.
SPPED, Instructional MOdule.3000, New Yotk State Education Dept:,1972., William Garth, Richard Allen, Rdbect O'Reilly.
3
Cr
771-
WI. Mastery Testing Component of ,BEST. Masary'tests (see Ap-4
gebd1X D) are a necessary extension of the ;trend, monitoring. Trend
Monitoring-tests on few (maybe one items per Objectia.''M4Stery.
:Testingtests many items .on an objective (5-100:or more). Mastery
Testingas indicated by its:name tells a lot abouta little bit
(one objectWe).- The-uses of Mastery Testing are:,,
a.. .for additional inforMatio6 - student performance on trend
tests does'not satisfy teacher that student really knows
or doesq't know an objective.
b. 'fcmconfirmation., Teacher wants to know the extent to
which the-student will-perform.on an objective.,' Trend
,
data will ,giVesoneoy't of one test item correct or In-.
correct for any objective.
Mastery test will- report 6/8 or 8/8. or 0/8 correct. The
k
teacher gains added assurance about the level 'of student performance
and the reliability of his performance on a particular objective,'. .
In -Brentwood .MAStery tests are in the" storage rooms_ or media
centers of each elementary scHool. A feacher aide on demand Will.
f.-
run off a ditto of a mastery test. Teacher will administer the
test. The aide may correct the test, using the answer key. The
results are immediately forwarded to the, teacher. for istruptiOnal
decisions based upon both trend and mastery data.
0
-72-
VIII. ,Implementation of the BEST Program in Brentwood,. In 1971
Dr, Robert O'Reilly, Chief of the Bureau of Research and COlturall
Affairs, New York State Education and D. William Gorth of the
University of Massachussetts came to Brentwood for a dinner meeting
:
with Dr. Ar4hdr R. Weger, Assistant Superintendent and with me,
Director of Curriculum K-12. They had a syktem they/ were trying'to
sell the Brentwood Schools and they were aware':of many advantages.10g
of working in Brentwood. First:ithe:size of the school district.
With 22,000 -gtudentg'at that time,, Brentwood tanked as lar
o
after the big s'ix cities. ,Secondly, the Brentwood Schools had de-...
velopedalreputation for'ihhovation in curriculum.- A:six...year Ford
Foundation School Improvement Grant (which this writer-directed in
1966-1967 and 1968) had resulted in fhepublication of several
programs which were disttibutld nationally.
The CAM system they,described seemed hopelessly complex at
. 7 .
that first meeting, yet, there was a great-appeal in the sYstem-
atization of'information.that was offered. 1Dr..-Brieger. and I were
aware that our scores in reading and math shad been declining.at a
rate greater than the state average. We lead also become caught.1: .7.
in
.budget squeeze's which forced the elimination of superVisory psitions.
Our class size was (and is) the largest in the area-fi Even before
we were fully aware of the understanding of potential of.the
program, 'there was an unwritten understanding that if we pdrsued
our interest at all, it-would be with the committment of very little
4.
81
0
-73-
.moheyi., We were assured that if we qualified there would be finan-.
INial support from the State Education DepartMent and from various
government sources.
I had been CurriCulum Coofdinator since 1968. In those
earlier years, we had coordinators of Social Sibdies, Reading,
Science, etc. Since those positions had been lost andme still
had twenty schools', 22,000 studentsand important curriculum
problems, my first inclination was to try to find someone else to
pursue the information given to us by Dr. Gorth and Dr. O'Reilly.
I went to Albany for further talks with Dr. O'Reilly and he provided
me with, various descriptions of the system. Slowly the potential
for monitoring instruction became clearer and clearer. Though it\\
still seemed difficult to find the time, as I grew to know the
program better it became evident to me that ultimately this project
was definitely Ore of my responsibility, and further, an oppor-
tunity to improve the instructional program. Dr. O'Reilly's
definition of Curriculum as "the set of behavioral objectives" at
first seemed.an overstatement and yet there was enough of a "truth-
ring" to,the statement that it diScouraged.my assigning such fund-
amental curriculum work..to some other administrators. For several
weeks l studied the program. I even tried explaining it to other
people '7" wi-thout.any 'success at all. This is when it first.btcame
evident that what is really an increditly.simple conceptiop.becomes
totally tangled in explanation so that the uninitiated usually,re7 1.
8 2
. 4.
celve a' mesdgettht saYSsoMething-lie, "Thereareomany
Yariab1es;"Objtctive,s4-cOmOuteTs', numbers, etc. that.; l,'
.-real-1*understand this." was a great relief ioAisco4er that
only the project Manager had tdunderstand all the. edijoatlonat.i*.
,
plicatiOns'and onlythe compater.programmer had to understand ail.
. . 0 .
thkeomputer implications,. It was entirely possible for people. to
lise the systeMWithout-underStanding the-relatiOpships.
_ .
Dr. O'Reilly was good enough. to arrange fot a visit of Brentwood
personnel to two Newyork State school districts which were tt-.
tempting theitirogramr. 70..
Ch cSang the inaVihals tp go on this trip was the subjeci of
long 4ls'cus.siphs between the curriculum cOrdinator and. the assistant
superintendent. The people-selected forthe*triliv were likely to be..
the people who would try f.the system.:i we ever, procedded to thatu.
point. It as immediately agreed that all thost people whomould
accompany-. .
viouldJ'be pti ncipals.
In selecting three principaIS ftoM among fourteen; certain
" criteria had to be followed..
1. .They'shoutd be principals who had shown a willingness to
. . .
innovate.
2. They shplqd _be opinion leaders who could iNUence others
to attemOt%thelorogram, should' it succeed.
J
Theyshad to be principals from among our six principals
- ofarget.schools " the target schools beig.definedias
I
-75-
those with students from familiei,receiving aid to
dependent children:
If we weretO spend any money at All on such a Project- it.
would certainly' have -`to come fr ESEA TitleI.funds from the
Federal Government.' Given all'the needs anl restrictions three
were finally chosen, The.firtt choicehad been a math Specialist.
for training teachers whenma had.Ford Foundation money_forpmodern
math. .ThetecOnd-'was a long established principal who was a
4
1....,
known' opinion' leader.' The.tHad.
was new to ouncilsirict but with,
.
... r
many years of experience in other districts. He often Wad demon-
stratecLa willingness to innovate. Each was approached, 'The
program was exptainecLbut her.e.:ivas sudden surge of enthusiaSm for
the idea of -a monitoring program. However, they agreed togo to
BallstOn,Spa and to Greece,..New York:
,
On this trip,:the decision to go ahead and attempt the.program
was really made, though it took Board of Education action at a, .
later time to make it official. The- first 'schoo'l was visited -
Ballston Spa Middle School. wouldhave been difficult to find
anything that resembled our own school district less. It was a
very large, modern, expensive building to house grades 6, and 8.
It was constructed for large group, small group and individualized
Lnstruction in open space: The various-akeat were brightly decor- 0
ated.:fThe staffing patterns were even.more .removed from our ex-.
perience. -A team leader, or grade leader worked with otherinembers
84 ."
,.......e.......en,......................y.........'. . . e .
.
. -76-
of the staff to develop.Objectives in each subject. Tests were
Constructed. Upon completion of units students were directed
to the test area where tests administered.and scored by aids.
CAM tests were sent to the Data Processing Center'which produced,
the reports for each individuel teacher. Everything seemed to be
,working very well and yet the impact on our group-from'Brentwood
was negaticie. Everything was so new and expensive; so much man-
power was available that it seemed to us that these must be the
factors that made CAM "go" in this schooL4istrict. It was for
tunate that;Dr. O'Reilly had arranged for us to visit Greece, New
York as well. In Greece, we still found a better personnel sit-
uation than we could have. Yet the project seemed to be getting
started in this district which in many ways was not too far re-.
'maed from our reality. The teachers and admLnistratori we tal,ked
to were, very, positive_ about this program for improving student
,performance., On the way home, we discussed how we go about givl-ng.
it a try in Brentwood..
The natural .step was 'to get permisSion to have some people.
trained In Comprehensive Achievement Monitoring. It seemed natural, -
and necessary to tnclude teachers in each of the three buildings
where the program was to be tried. Jutt,as naturally- -we thought
of those threeteachers as Ap team leadert-to help initiate it
in each of the schools, The same pitefeUitttet seemed. to apply, to
.
the.te4Chers'who would help this pogram'succeed. They needed to
,be innovative and to be opinion leaders in each of tAie buildings.
"77-
'Once the teachers were selected all six people went to a one week
workshop at the University of Massachussetts. Dr. William Garth
conducted intensive, all day programs which. described, demonstrated
and simulated the CAM operation. Our three principals and three
teachers returneci to Brentwood as authorities in monitoring in-
.sstruction. At this'point we.had enthusiasm and knowledge and
7 nothing else. Yet, we decided to implement the program...in
order to implement the program we would need-objectives, - test
items, tests, guides, answer sheets, etc. etc. To get organizedj.
'for producing and later carrying out these tasks. we described--the-
---following organization. (see fig. 22)
It was decided that the Director of-C-Urriculum would be the
project manager. The director would report to the assistant
superintendent who in turn would report all phases except evaluation
to the sUperineendent. The administrator for evalbation (at that
time on staff for ESEA) was to report evauatiOndirectly to that
'superintend'ent. The director would assume responsibility for all
. relat.iont Interaally with priptipals, team leaders, central' office,
students district curriculum committees, Dataj4rocesOing Center and
the State Education.Department.17.
At this time we also carefully defined the team leader's role.
'(see fLg. 23) /The team leader would carry on dOrt.liaison be-
tween faculty arid principal. He would copduct'in-service workshop$
4 .
for interpretation of data and explanation of process.
°
.
778-
PROJECT DIRECTOR'S ROLE
PRINCIPALS
TEAM LEADERS
0 ff
SUPERINTENDENT EVALUATOR
ASSISTANTSUPERINTENDENT
DISTRICT MATHCOM PUTTEE
FACULTY 4 STUDENTS
DATAPROCESSING-
CENTER
. ADDITIONAL DIRECTOR DUTIES!
SCHEDULES 4t TESTING, REPORTING.2. CONDUCT MEETINGS - AG6DA, MINUTES.3. REPORT TO E3OARD OF" EDUCATION AND PUBLIC.4.. 'ASSIST SCHOOLS
O.
STATE ,EDUCATIONDEPARTMENT .
A. IN EZVICE,F-OR4ACULTYB. OR I ENTATION FOR PARENTS
%
SUPERVISE PRODUCTION TO QUALITY CONTROL
A. OBJECTIVEB. ITEMSC. TESTS
BOCES
_
d.
DISTRICT READING ,
COMITTEE
1W.
b
(fig.22) -. Chart illustrating relationshipsfor project director
8,-
S.
a
SLPEPIN'TENDENT - ASS 'T SUPERINTENbENT
11\ 7.DIRECTOR
DATA PROCESSINGCENTER
ROLE!
:
a
. 79 A/.
TEAM LEADER'S ROLE
TEAM LEADER, STATE EDUCATION
DEPARTMENT .
FACULTY
VSTUDENTS
0 .1
PRINCIPAL
PARENTS
1. PROVIDE TESTS, ANSWER SHEETS AND 'PRINT OUTS FOR TEACHERS AND STUDENTS.
2. LEAD TEST PERIOD DISCUSSICN WITH STAFF.FOR: .
A. ANALYSIS OF DATAB. STRATEGIES FOR CHANGEC. MATERIALS, PRACTICESW. 04-GOING RECORD OF DEFICIENCES IN OBJECTIVES, ITEMS,'TESTSE. ATTEND MEETINGS AT DISTRICT LEVEL TO SAfIECONCERN, MATERIALS,
PRACTICES
.
Mgr 23) -- . Chart Illustrating reatiOnshipsfor team leaders
88
-80-'
On. their return from the University of .Massachusetts,' the
three teachers, now teams leaders and three principals began an
intensive effdrt to produce the material, needed for-testing in the
cqming year. We stockpiled all, available banks of objeCOves
test items for them to work with. The 10X bank; The Downer's
Grove, Pa. bank and the State Education Bank were all very useful.
However, the work done i Greece, New York was more helpful in the -
initial trial. One of.th" early and key decisions made was to test
.,the existing curriculum rather than revise it..To dotiiat, the .
members of of the team went th ugh our 'text books activity by activity,
\
to find the objective that sh uld.be implicit in each exercise..
Once listed by- grade level thetask,became the -writing of at-
least'five test items of'equivaent difficulty, reliability and
diffi ty. Since a'great many-m th tests were available and since
given a model 4th, 5th and 6th grade math test items, .similar
items can be produced- great profu ion. This part of the task
proceeded Smoothly: Noril'Of fr rs who would Use this4
monitoring programhad.(to our knowledge) ever heard of such pro-
grams. Therefore, it became' necessary to write a guide for teachers
to help them use the material. Students and arents also had to
learn so a guide would have to be provided or them as well. To-
gether with answer keys.and tests, all the material would then have
to be coded for the computer.
In September, a joint faculty meeting was held for the three
sChools who, were going to participate in the program. A team from
the State Education Department came to conduct an introductory sen-
tence. About half of the 4th, 5th and6th grade teachers attended.
The attempt to explain the CAM system in an hour and a half.was
too ambitiouS. It wa.sa very warm day In the cafeteria of Northwest
Elementary School. Teachers quickly became impatient.with the
complex systemsthat were being presented schematically on acetate
after acetate.
This. was to have been-the "kiCk-off", the initial step in an
in- service course of about fifteen hours for training teachers In
ComprehensWe Achievement Monitoring.
When this session was concluded, I held discussions with other
teachers, principals and team leaders and, members of the Bureau of
Research and School Affairs. With little support, I made the de-
cision to omit, eliminate, skip and otherwise avoid the in-service
education part of the preparation for and implementation of CAM..
This turned out tobe ake dec,P.
made for the foll4W pg reasons:
ion for many reasons. It was
,
1. .1 knew there was a national objection to accountability4
developing in, teacher groups.
2. I knew that a previous attempt to implement this program
in another large Long Island SchObVdistrict had been re-*
jected after' extensive in- service education.
3. 1 believed (with Socrates and Dewey to pick some desirable
910
-82-
company) that.the teacherS would Clulokly andoalmOst ef-
fortlessly lern the system if they would only.administer
the tests and then work with the students to interpret them.
This is the way it has developed. We now have hundreds of
teachers who really, are qualified in operating with a monitoring
system. They.beCame qualified by getting and using information in
. a very basic, immediate way. They mightjail on an exam question, .
dbou e technique for randomization of student group's or test
items (which our computer operator has td know) but that has
hothing'to do with helping their students learn. Now that the pro-\
gram has been operating successfully for three yearS.with hundreds .
of teache s*and thousands of students I often looklmck to this as
one of the most important dectsions'cOntributing to the program.
Obtaining CompUter<Services: Though: Brentwood had a computer
'since 1968, there practicallymas no-rnstrucfional applicators made.
it was used for payroll, scheduling, and,report cards. The Bureau
of Research made some contacts to provide us with computer processing
Of-the program. Fora charge of' 25C per pupil they were to put up
the banks of information and-generate reports for our schools. At
several early meetings.the inability of technecians and eduCators
to communicate nearly aborted the prograM. Thequality of the0
perSonnel which represented Grumman Data'Systems:was prObably
intimidating topublicischool people. Some of them had just cOm-.
pleted work on government contracts which included the "man on the
moon" project. A great deal of patience-eventuaily previlled,
set of forms was devised:and objectives, items and student in-.
formation were painstakingly coded for, the computer. One early
'probremto manifest i
Grumman Data Systems Ad
I f was that "of the" 20 mile distance between'r-
Brentwood.. The telephone was often in-
ade'quate.. Much time was taken by driving. Eventually all of the
information had been, put up on the computer and 'print-outs were
delivered flOr pi-60f readlyg4.. Here we began school districtS real
introdbction to "qUality Control" because even when we proof read
carefully, mistakes/Were made. in the correction and coding prodess.se
.
These mistakes, of course, returned in the final tests and student
lists. But in early October, we were ready to test, we thought. .
The Fir Test: In this first trial thel=4 was an answer sheet
provided to each student but the student had ;cs print his own name,
number, teacher name and number and homeroom etc. ,Then he would
take the test.' Special IBM pencils were required for marking ,these
farms. When th'e students had completed thetetts,,the answer
Sheets were collected and taken to' Grumman-Data Systems for scoring
and reporting. We awaited the results which were supposed to crime
in 48 hours; The'first test took more'than a weeVand when he
4
results beteme.avaIlable we recognized that'many problems:had yet. ,
. .
-4ta a,be solved. The most important, was accuracy. Several.
I
-things, iti
seems, have to go wrong when 4th, 5th and 6th grade
students write down information by band.to be copied by key. punch
2
I
- 84-
operators.
a. they make mistakes
b, some of them havehandwrIting which defies anyone's
reading ski 1-1
c. some key punch operators make mistakes.
'As long as we worked with Grumman Data Systems'we never really
solv4d this problem. Through careful checking by team leaders and
ails, we reduced thenumberof.errors.
We neverhad enough IBM pencils. They disappeared at amazing
rates. Wcut them in half to double the numbers but still couldn't
-..keep up.
Meetings were held to try to improVe quality control, It did.
t<iraprove,but never to a satisfactory
.
level.. , \
,The turnaround problem-threatened the program most severely.
.There isi need for immediady in.returning,results- for teacher, and
.. ,
stUdent's:use. 'Last week or two weeks ago 'seems much too 'remote.. - .
...
..
. ,
..
,
, Despite.numerous little and big events that were not supposed"
.40 occur but did with regularity, the teachers -`and team leaders'
maintained a posittvecattitude about monitoring. An important.
factor may have been that coupled to the complete omission of In-
service iqstruction we had repeatedly announced-to the teachers
that the program probably woUldn' work. We told them that.our.
expectations for the first year were that we would learn to.give'
0 :
the tests and interpret the results even if those results were'not
-.1
tl
.-85-
'what they should have been. This approach wag taken bOause we
had had sufficient experience with programs that promis magic
cures thayle decided to minimize expectations so that.,..real
,
productivity in the progran would be viewed as a pleasant surprise.
Describing the possibility of failure and the expectation of much)
.
less than total success helped us with .the faculty. They'appre-
dated candor and realism in educational innovation.
Throughout the period the State Education Department main-
tained its involvement with us. They had contracted D :.Shelly
Harrison of Technovations (Later - Public Systems Research) and
Stony Brook University to be a liaison be'tween the Brentwood Schools
and Gru4iman Data' Systems. Members of Dr. O'Rei ly's staff visited
regularly. The Bureau had contracted Dr. S. Alan Cohen (later
'author of Random Hous e High inteniity Program) to develop objectives,
and test items in reading. A steady-Stream of preliminary work in
reading keg' flooding the curriculum office. The'math r;onitot.ing
program was being praised by students and teachers 'and it seemed ,,
desirable to monitor reading as well since ourcores had been.de-,
...
.
dining.. Besides, we now had' a,staff of teachers and team leaders
. ,
teained in monitoring system. The system wouJd'be the same no,
matter what subject was monitored.. BytheSpring.i had announced
that two.schoolswould begin monitoring Behavioral Objectives in
'reading. The BUreau Of Research had promised an extensiVe set of
4
.jobjectives with many validated test items for each objective.
.
00
-86-
.
it was, to hear it described, merely a matter of choosiog the
t
material and installing the system.'
_Also in the Spring: at the conclusion of the first year,
'Grumman Data Systems announced that its rate would increase from,--
.
254 per pupil to pearly 0 4 per pupil per test.- Since we were : -
planning to test in-reading as well, thisoiNoUld. hav6 Meant the
cost of $1.20 per pupil per test period,' if,,,students- were in
0
-'both programs. -This was more money itia We were.Oending on-text,. .
. 0
books and supplies. Since we couldn't real affort.the twenty five.
cents charge in the first place, the new cost was absolutely out
\.
. .
of the queStion.- .
. ,
In discussing-this latest problem withDr. Harrison,: he.in-: it" :, ,,
ts,
formed me that lits orgarfization, T chnovattons, could, for $4,000.00
-
program our district computer'to perform tasks at
much 1 ss,than25 per student.
le kimultaneously-r in soap-opera fashion, DT. Nannint, the
*-Superintendent, annoUnced-Kis resignation effective. in June. Dr.
.Nann.ini was reluctant to authorize the progrmMing laccatise,:he
thought that such adecision should be made by the new superin-
tendent. However, if our computer was to be programmed by September,
immediate signing of the contract was necessary. I signed the =T-.
tract without authority to do so. My assumption was that-improved ,.4
student performanceld 'mitigate in my fdor,.if not justify the
act: Dr. Harrison and. his crew from Technovations began the work
9.5
00'
-87-
in thq summer. From the beginning, Mr, Joseph llotolo, the school,
district's Coordinator of Data Processing said that the program in
Fortran IT would '.
a. Joe too slow
b. not Wow for all the students that Brentwood would have
to.put in the program if the program Were to succeed In
both Math and reading."..
In the meant1tre, Mr. Di Pietro was made Superintendent. I Mew
that ;t would generally be his view that'anyone who Oommitted,theo
school district toa $4,000.00 contract without. authority' should'be
fired -- not an altogether unreasonable paint of. vieW.
To compound the difficult and complex, I received a. telephone
call from_Dr. O'Reilly informing me that the objectives and test
items and financial support would not be available for installing .
the reading program to which I had cmmitted myself,. thedistricf,
and the Data Processing Center. If the first year.of CAMgmth
testing had gone well, it's ending promised to be an academic;
social, and personal disaster.
Things could hardly'` have been worse, since there was also a
recession and a shortage of,,lobs. that SpringL---:
In they- 'Spring of. 1972 the District Reading Committee was ready
to make its recommendation to the District CurriculUm Advisory
Council the Curricblum Coordiriator is chairman of this council
which recommends iprograms and materials/to the superintendent of
1
.-88-
schools. This program carried a set of behavioral objectives in
reading comprehensiOn. Even with this adoption there would not have
been sufficient time to write all. the test items. However, strat-
.
egy,for extrication began to,form. I called, the salesman to Y
office and told him that I had heard that, the reding'coMmittee
Was going' to recommend his series for adoption in Brentwood. He
indicated that he had heard similar rumors.himself. , I then ex-,
plajned to him that as chairman of the Curriculum Advisory Council,
I couldn'l possiblyallowsuch a recommendation to heard, much
less sent to the superintendent if the program was not accompanied
by four test items for e;ach objective. He took the news well and
recovered quickly: He called his main office. There a vice presi-
dent called an editor in Denver and within an ho.ur,. 33here was an
agreement to provide test items if the program was .recommended by
the,Reading Committee, the Curriculum Advisory Council and the
Superintendent. Now there was a slight hope that the reading
monitoring program:could be delivered as promised. In the meantime,
the original CAM math team was going to work for_two weeks to
rewrite objectives and test items which had demonstrated flaws in
the first year. This was to be a major strength of the program,
that in a period of time from constant rewriting there would emerge
a program completely tailored to the Deeds of our school district.
As the first b411 from TechnovatiOns ceme4due, I arranged for. .e
payment with ESEA money that we had planned to use to pay Grumman
-89-
Data Systems but that amount was now exhausted and no provision had-.
yet been made for the-final payment.
By the end of the suwer the -publisher, true to his bargaim
supplied four test items for each objective but they were not in
usable'form for monitoring program purposes. Their chief defect
.was that nearly every item required multiple biehaviors-on the part
of the students In a panic the last week of the summer I began to
rewrite the test items. For the next six weeks I wrote every day.
.
In ,early September we began coding the math for scoring on our own,
A
computer.7-
Another bit of rescuing as the Federal' Projects Evaluator in-
k
formed me and the Superintendent thai the'students in the monitoring
program(see evaluation section) had in fact made greater gains than
the Control group. This publication gave added,.impetus for going
or with the:math program. After publiCation of the results, I'
submitted. the last bill to the Superintendent with1
a on explan-
aton which'he accepted. However, many of Mr. Rototols pre-
dictions came through.as we were through the first semester without
testing in. reading.
The program on our own oomputer.was slow. 'We were ni:ot gluing
the rapid turn around we,had been promised. The'program worked
only with regular attention from the staff of Technpvations.
Around mid-yeareof the second year of CAM (Renamed B.E.S.T. by-°.
D .'Harrison as he programmed the corplicer) Mr. Rotolo inforthed us
r
V
that if he rewrote the entire program, using COBOL,computer lan
we could enter all our elementary students in at least four >ro -
grAms.with over-night turn-around if tests were carefull scheduled.
At about this time dissatisfaction with'slow-tur -around, errors,
lack of reading program had caused principals and teem leaders to
be less confident of the programs future. By ow, though, the
Spperintendent had become an ally of the program. He called a
meeting Attended by all) interested art'es in which the !following.
agreements were made:
1. Rewrite program in Cobol
2. Try reading in'gradee 5 and 6 In two schools
as soon as possibl .1e
3. Set a schedu4e/f r spreading progra m to all fourteen
elem e'ntary sc ols in'an orderly fashion as long as
improved student performanceTistifiedthe effort.
Dr. Harrison d his crew were finally ready to put up' the
banks for reading monitoring in early SprIrig. TwO principals who
had served on the BEST reading committee were ahead with ipstal-
lation in'their Schools. The reading consultants became the team
leaders. Again me used many of thestrategies which seemedtp.,1vork
in Math. We conducteid no in-service ,Workshops. We allowed the
teachers- to learn to use the syktemibit using' the systeM. A pre-.
'liminary evaluation after only a short trial was Ve'ry, positiVe
about the effectpf BEST on student achievement.
A
a
99
I
a
We concluded 04 year'4n a very cheery note compared to the
feelings of the preceding summer. The teams were again-or-
'ganized to rewrite and refine the prograM. ,,.One major change was
needed though; we could not expand the program with the program
installed by Technovations. Mr. RotOhnow. proceeded to write
and, install his own COBOL version of the program. This work went
on-alr summer.
In the meantime,.,t6 reading team of teachers and admin-,
istrator'S were writing manuals (see Appertioix. K)§ guides for parents
and student, cross reference guides ,(see Appendix L) for Materials,. ...
, .
. . .
reading objective bank (see'Appendix M), test items bank, (see
'. , .
Appendix N), and tests (see Appendix b). . .
In rewriting the computer program,\ Mr. Roto19)developed
,
another refinement. The computer would now print all tests of'
dittos '(see Appendix 0) for distribution to the schools. This
would now take us out of the printing busihess---Which was a',
rather large part of the effort. High school girlS and"other.
temporary help were recruited to do all the coding for entering the
millions,of bits of information for both the reading and math4
program's.
We began year three of the prograM very enthusiastically.
There were slight delays in testing as final computer adjustments,
were made.
Right from the beginning the nevoprogram worked with an
100
-92-
ficiency that waSa marvel' for everyone who Was'involved. Tests
were scored and reports produced.overnight. Test answer sheets
collected'by two o'clock in the afternoon brought all reports back
to each school by nine o'clock the next morning.p
Another feiture that Mr. Rotolo built into_the programsoled
90% of our/quality control problems. It was the printed answer
sheet (see Appendix P). The computer printed an answer sheet for
every student with all information (names;numbers, sections,.etc.)
except the answers to the test . The only writing now done by,
studentt was the simple mark sense fill-in. The Brentwood Op Sean
also permitted the use of regular #2 pencils -'solving another
test problem.
The program went soismoothly that by midyear the decision was
made to add two schools to the'math testing prog am and in addition
to add the reading testing program in the original BEST math schools.
Studies covering student progress in each of the subject'werein'r
progress. The biggest success of the program was now readily evident
as parents came to meetings to learn how to interpret the test rer.
Sults that their children were now regularly bringing home.
, In the Spring I participated' in a statewide workshop in Albany
to describe CAM and interest other districts. Brentwood had succeeded
in developing a full computerized monitoring system in'math with
ll
0'
state cooperation. Beyond'hat,ion its own, BrentwoOd had developed
a reading monitoring system. At this writing in.1975, Brentwood is
still the only district in New York Stat4.,withta full monitoring
10i
. .
-93-
program, comple ely computerized in reading. The Curriculum
Coordinator t embers of the Brentwood staff have now entertainedry;
hundreds of visitors. We halve gone to many other school. districts to
help them implement math programs. We have proVided our objectives
and test items to help people get started in reading as well.
The hypothesis that this program would help students achieve
better results in math and reading has been confirmed. (See .ev-
aluation section)
An added bonus has been the,affect development brought about
by focusing on instruction. Letting everyone know what we amt.
trying to do and letting ever ')one know how he individually is
succeeding seems to serve as an Important motivational device.
As I once\remarked to a teachers group, no one has ever met
,a person who gets up in the morning and says to himself, "Today
I'M going to do,a rotten job." Bat for some of us it works-out
that way because we don't have enough information about the kind. .
Jof job we are doing. Providing information, or feedback as buy/
(computer specialists choose to call it, makes postible the regular
course correction for both teacher and student.
J 102
V
-94-
IX. Evaluation'. To quote Dr. Daniel Stufflebeam, "Evaluation is
in fact a procesS7which can /provide key inputs to aid planners and
decision makers in making necessary decisions. Evaluation inputs
can provide these data not just at the end of a project or activity
but throughout all stages, from initial 'drawing board' stages
through operation or implementation to termination or long range
continuation."6
This definition has probably never been more applicable to a
project than it is to our experience with BEST ,- CAM - (BEST is
the Brentwood variation of Comprehensive Achievement Monitoring)
Context'Evaluation
Q. Why did we need a monitoring system?
A. Because student perfo.rmance'ha hown a regular decline.
Q. Now much could we afford?
A. Little. (see Budget) We financed through ESEA.
Q. Now much could we commit to long term expense for maintaining
the program?
A. Little. (see Budget),
Q. What experience did we bring t the project?
A. Extensive work in curriculuM development. (see Implementations)
6Stuff)ebeam, Daniel., "The Process of Evaluation An.tasy Example"Unpublished handout distributed at Cluster meeting in Education,undated.
103A
"1-
95
What was the monitoring system that was needed?
A. One that would perform clerical task and provide
usable informatiori. - CAM only one available which
met the criteria. (see Problem).
InpUt Evaluation
Q. How long did we have to find a solution to declihing scores
in reading and math? ,
A. We needed a solution as quickly as possible. The downward
trend was threatening to become a collapse.
Project Plan and California Test Bureau's diagnostic math and
several other systems were compared. None met the criteria set
in the Context Evaluation. The input and the context evaluations
both suggest a rapid move.
Process Evaluation
From the very beginning BEST was an outstanding example of
an innovation which produced'its own regular process evaluation
for individtels.and groups (see Program). It even produced item
analysis (see fig. 24) and error analysis (see fig, 25)., By
definition this monitoring program is Prosess Evaluation.
Product Evaluation
In bctob 4 I testified before a Panel of the New York
State Office of Performance Review (Rockefeller's Klepak,Cammissipn .
watchdog on education). The Commission task was to evaluate test-
ing procedures in- New York State. In my presentatibh; I mgde as
104
ITE
LAtini$19 F
OR
MA
TH
AS
OF
11/11/73
-IT
EM
C1401 O
E S
_N
0EB
C
PR
EP
OS
TP
RE
PO
ST
PR
EP
OS
TP
RE
PO
ST
PR
E,
PO
ST
.P
RE
1P
OS
T'
.$1050100101
._1OS414014?
1050100103
30
-
a5
0..
_....3
____ 3_
6_ ___
8,._
._
140_ _.142
4039
f7ii
11079
3.
4.i
0
201
41i2
103
80*9
520
160
0
1050100104
___/.050100301-.
32
37
14....
12
ta.
_121
9815
115
10.
_..0
.P.
:4
a-019
363i
.
9171
i7a
0
01-
105010030228
3122
10014
1012
419
111'
0
...
1050100303...,
---1050100304
20'
:21
250
4..24___
13_.
14___
_96 __79
2314
..1
..
37
17 -11
1115'
0
00
105010040128
,123_:_1002110
29-
553
1266
2692
121
6x
--1050100402
-____1050100.4133
..
32
'3i,,.
.0_64.
_ 26(,______93 __16_____66__._15
.____27.____7___
.0..._ ..__ _ .
...
....0
32-
11'- -
256
41115
287'
1
____0_.___._.___
00-
C4
4
1,47.- r
CI (13
.... CT
^14.. <......
E-
i
I%
.01050100404
ON
240
.i2512
4730
52'
828
100
__11050200'101
19...
'17
.0......It __i
_____ ..9' *_____ 5__________36.___5._____142e V
_ 00.
4.)
.__
.. _.
_ .___
.._.
_____
_______,_--
129
16192
135
11i
O
_0-. --
-4a-
.-...--10502..001_02
1R&
0200103
..
1.51
73
133
16291
17.1Q
11:
1050200104
_1050200201
1.050200202
.1227
0---.3
.-3.
10_3
/6.0
0-
26
1144
7643
164
41
151
17,2
103
17480
22'12
00
1050200203
105020020
337
25
10
0_158 _86_
.1-
910
2129
81'
105020030133
210
443
7.114
4246
112
- -0'
.
105020030229
36
639._
0-.2
.._
_42__
_._
-_
-3
11755
495
244
494
0
1059200304'
10:602040'1.
1050e0mgi01
48.4
354
.121
54_
39
4.
23.
-4.2
0
3176
315
1131
55...
6030
0
6 "._34n
L:725
220
0
-
.S T U D E N T
pERFoRMANCE
NALYSI-S
B Y
C L' A S S
_D
AT
E: 204.
...:-.-...-, TEACHER NO: 534
_-...---
-SUBJECT
2MATH ....- ......
...''
_-
-.
TEST NO
:3
-- -----
COURSE NO
:30.0 --a-.----;-- -
SCHOOL No
03
08 J E
6 i rI V -E
S\--.
--- -
F - c---
11
11
11
11
/1
11
11
11
11 11 1
11
1 1.1 11
11
11
11
11
11
1C
5 5 5 5 5 5 5 5 5 . 5 5 5 5 5 - 5 5 5 5 55 5 5 5 5 5 5.5 5 5 5 5 5 5 5 5 5 5 5 5
4-12 2 2 2 2 2 2 2 2 2 2 a 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 P 2 2
-0 0 0 OA
-G-0-0-0-0-0 0 0 3 0 0 0 0 0'
0 0 0 0 0 0 0 0 0 0 0 0 0 0.0 0 0 0 01
111 1-1-2.2.4-3-3-3-3-3-3-3.3 3 3.4 4
4 4.4 4 5-5-5 5 6 6.6 6 6 6 7 a 8 8
o, 0 0 0 0 0 0 0 0 00 3 O
ro 0 0 0 0 0 0 0 0 0 0 0 C 0
0 0 0 0 0 0 0 0 0 0 0 0-000000'000000000001100000000000000000000
> a)
4 5 6 7 8 9 3 4
1 2 3 4 5 6 7 8 9 0 1
1 2 3 4 5 8 5 6 7 8.1 4 5 6 7 9.2 2 3 8
_4.,
-W
-W---C
- C--
C C
-1,1 C C
C C
W - -C
C1-- C
--- C----C
--W----
C-1.1 W
--w
14 - 14STUDENT.NAME
LAMARCHE
LUIS
LEM
ON
.0 0:
>.
RO
BE
RT
WW
WC
CC
ti NC
CC
CC
CC
WC
1.1C
CC
CC
CW
..
TI T
TT
TT
T-0c
(I)
----LOPEZ ------- D,EBRA ----W-W-WW
C- -W--- C -W -
H.
C-1,1
--C -W C -W W W C-
VI
_u..-a)
.---.
0tn
WC
CC
CC
W C
CC
W C
WC
C W
CC
CC
CC
CC
>.L
TT
ITT
TT
T T
Tcn
a)- R
IC
t.
--
-....-
v-
C C
W C
W C
C C
C-------
C-C
CC
CC
C C
C-C
C---41-----
TT
TT
T T
TT
T-
-T.
0_ -a
r...MCMILLAN
KIM
NCLAN----- ELIZABETH
t..
.
PC
i;NE
R.
'rriOcY
mA
SW
WC
CC
CC
C C
CC
W W
WC
CC
WC
WC
WC
WC
1.4in
TT
TT
TT
TT
TI
ot.
-' - - ----44.*:
W W
CC
C- C
C C
- Cc
-wc
c--- c---c wc--w
c C.
c-- c c w-W
T.-T
T T
TT
TT
T---- -T
------ -D
ON
NA
._
SE
IcER
TJA
ME
S
..
CR
CC
CC
CC
CC
CC
WW
C C
Cv1
CC
C C
Cw
s
--- 0/ ANE
WC-C-C- C- C- CC-C--C.-------CW-C-CC C-C C- C W C CC
CC
T ,T
TT
---T
T
....
-- VUTURO
WILLIAM
C C
W W
CW
CC
CW
CC
..CC
WW
W14
VI
cwc
ccW
iq
.T
TT
TT
1/
TT
T
--WA
VE
------
EILE
ICN
-.-- ---.---W W
---C C
-------C-C
.CC
C -it
CC
.- W C
- W ---C
-- -.c---.-C-C
---C--C
- C -- - C
W---- W
--,-- ---... ....._ .. _
SCO
RE
g''C
CO
RR
EC
i,W
WR
ON
O)
NN
0 RE
SP
ON
SE
, TP
OS
TIN
CT
r TIO
NA
Leir
41.....1..
r
strong a ease.as I could fbr the use of CriterionNmeasures to
judge student success. We had amassed impressive statistics in
Brentwood to support this point of view. The most appropriate
product evaluation is the following statement that I made on the-,
behalf of the-Brentwood Schools at opening hearings held by the
Office of Education Performance Review for the New York State,
legislature. They' stakistics were compiled by Mr.
.AdmJnistrative Assistant for Evaluation.
-%
0
9
David Holt,
-99-
STATE OF NEW YORK
EXECUTIVE CHAMBEROFFICE OF EDUCATION PERFORMANCE REVIEW
STATE CAPITOLALBANX 12224
September 24, 1974
Dear Dr. Fournier:
Thank you for agreeing to participate in the public
meetings on pupil testing sponsored by this Office on Octo-
ber 3 and 4, 1974 in Albany. The meetings will begin each
day at 9:3D A.M. because of the large number of individuals
speaking.4s. "
We have scheduled your presentation for October 3
between 1:30 P.M. and 4:30 P.M. Following your formal
statement the panel members may want to discuss sorrie of the
points you aise. Please bring eight copies of your state-
ment wit yo . Enclosed is the schedule of speakers.'
Observer are welcome po attend.
Thp meetings will be held in Senate Hearing Room A
in the Legislative Office-Building (corner of State and
South gwan Streets). Enclosed is a map showing the loca-
tion of the building. Please use the State Street entrance.
One of our people will be in the foyer to greet you.
If you have any questions, don't hesitate to call me
at q18) 474-3342 or 474-3170. We are looking forward to
your participation.
'Dr. Raymond Fournier'Wrector of curriculum-Brentwood Public SqdolsThird Avenue and Fourth StreetsBrentwood, New yorl:' 11717
108
Siricerely,
. -100-
Brentwood Unbn Free 'Schaa-DistrictBRENTWOOD, NEW YORK 11717
MO 435-2123
Daniei Klepek, Dir4ctcrOffice of. Education FerformanceReviewAlbany, New York
t
REPORT ON THE UiPACT OF ERENTW00D1S
CRITERION REFERENCED TESTING PROGRAM
Submitted by:
Raymond FournierCurriculum CooNinator
G. Guy 6iPietro Superintendent
Arthur R. Srieger Assistant Superintendent
Rayerorid Fournier Curriculum Coordinator
David Holt Administrator for Evaluation0- .,
-101- r
. .
IA 1971; the Brentwdod Schools began a trial of ComprehenStve:
AchievementMonitoring7in cooperation with4pr.- Robert O'Reilly and the
Bureau of Research and CultUral Affairs. Comprehensive Achievement Monitoring,
most Simply described, is a system to,
a: specifying the.19arning expected from the student d6ringa semester.
b. testing the students six times during the semester..
C."preporTing to parents, students; teachers and adminiStratort the
progress that'js being made and the degree in wtrich the learning
being retained. Using the computer to score tests and compile re
ports made it possible to handle the .mass of information.
There.were, many reasons for this move. First, and most .compelli s
that despite curriculum reforms, a variety of supervision strategies
materials, changes, and sttudent groUpings,Ichildren in, our schools.were not
making satisfadtoryscOres on'norm-referenced tests. We were in a situation
in which we didn't seem capable of changing student performance. on norm-
referenced. tests, yet we believed that we could teach chijdren to read and',
.perform in mathematics. Our experience with- norm-referenced tests was con-'
firmed.by_similar. frustration in school districts like ours throughout the
country.
The Brentwood Schools, have, more than 21,000 students in grades K-12.
There are twelve (12) elementary schools, a 7th .grade center, four (4) juniorm
high schOols and one.(1) high school. StuWiTs In grades7, 10, 11, and 12. .
.
.
are on double session. ,%
.
`.,!, .Brentwood's assessed valuation perpupil. is the lowestin..:Its area
Even with a high rite of state aid, Brentwood spends less per pupil than.
neighboring districts.. The student population is 77.9% White, 5.8% Black
and 15.8% Spanish surnamed.
tl
0 -tv47
Target Schools 14.5% aid to dependent children-
10.5% Black .
28.8% Spanish surnamed
Criterion referenced testing is direct and straightforward. The task
D
and performance expected are specified. The following example is from our
6th, grade program:'
T16-05,004-00 Finding averaces of who numbers -----
OBJECTIVE: Studehts will select the correct averagenumber of any given set-up to five wholenumbers.
EXAMPLE: A student received'on five tests the followinggrades:: 70,90,80,85 and 100. What was, his
average grade?
t(A) 90
(C) 70
(B) 80
(D) 85
First Semester
If the task is appropriate, success is achieved when the student solves
problems of this Rind. Being above or below. the 50th perCentile in some part
of the California Achievement Tests is rather irrevetant to the teaching and
learning task involved in this mathematics objective.
Starting in 1971 we speCified objectives and tests in Mathematics for
grades 4, 5, 6. In 1972 we developed objectives and tests for reading at' the
elementary level. We distributed the Objectives and sample test items to
students and parents. Computer reports on 'student perforience were provided
to teachers,. students and parents.
5tudents worked toward -14e objectives, and criterion tests indicated that
they .were succeeding. Performance improved' from test to'test.in bbth math and
reading. Teachers, parents and principals reported satisfaction that students
were learning. Materials and activiies'were eing organize
achieve the objectives.
1 1
(
Still the'questions were heard, "How will they do on ;ormrreferendedA.
tests?", "How- will they compare to the;norp?" 4A
it is somewhat a surprise that students working In criterion referenced
,
Instructions -have regularly Improved their performance on norm -referenced
tests since 1971: It placei us in the position of using norm- referenced
testing as validation for criterion referenced instruction, The dilemma
occurs because criterion referenced Instruction defines success at the per-
formance of the task white norms define success as a position relative to
other peoplewho took the test: 40
'While Using the data from norm-referenced'tests to demonstrate success
of our criterion ref renced instruCtion, we are uncomfortablbecause it'may
be implicit recognition of, the power and acceptance enjoyed by he.norm-,
referenced tests.' Yet, the fundamental, flaw of norm-referenced tests is.that
only one-hall of the population can succeed by being above the 50th percentile.
One-half must all, regardless of,theiinstruction. Regardless of the tasks
that students can perform, one -half the poputation_mustbe below the '50th
percentile on these tests.
112
-104-
.0 -:ART I is a comparison of mean scores of children in the three ESEA
target schools having a 'student population that is most seriously dis-.
avantaged to theJnean scores of the other three target schools. The
test used in the coMparisOn is theNmath computation subtest from the'
California Achievement Test, Level 3, Form A, 1970 edition. The complete
rath and reading tests were given to the children in all elementary
:schoolS in grades 4, 5 and 6,as part of the Distridtwide testing program
during May of.1971 and 1972 and February 1973. The graphs reveal that
.although the CAM group was significantly below the-non-CAM group in May- 4
of 1971 at the end of grade four preceding the introduction of CAM math,
by May of 1972 the CAM group had pulled abreast of the non -CAM group at
the, end of:grade five. As sixth graders, the CAM _group had pulled away
from the non-CAM group, and although the difference was not quite sig-
nificant in February, a significant difference was projected for the end
of the year.
113
7.0
.6.5
*Li 0,0,,w
5.5LinC
3a<7C
C
5.0
4.5.
4.0M
ON
TH
y6 9 10 11 12 1 2
a
10 11 12 L 2 3 4 5
YE
AR
7172
TIM
EA
M'73
A T
HR
EE
YE
AR
CO
MP
AR
ISO
N O
FM
EA
FO
R C
AM
AN
D N
ON
-CA
M P
RO
JEC
TS
CIN
MA
TH
AT
MS
CO
MP
UT
AT
ION
OLS
ON
TH
E C
ALIF
OR
NIA
AC
HIE
VE
ME
NT
TE
ST
a
-106-
.
CHART 2 is a compprison of a sample of stUdents,from a control and an
experimental group.in one school that introduced CAM in reading in March
of 1973. The Gates-MacGinitie Reading Test'Leyel D Form'l was given as
4
a pretest at the end of Januery,'and Form 2 was given as a Posttest at
the beginnlng. of June. Although the CAM was to begin the first week in
February, several problems delayed the start Until the last week of March.
) Only forty-two,instructronal days elapsed' between the first CAM test and
the fast CAM- test. After initial differences between the two groups due
to chance variation were equalized through analysis of covariance, the
posttest means were compared. As shown in the graph, the exper'imental
group improved ther reading comprehension more than the control group.
When the adjusted pottest means were compared, the difference approached
_statistical significance with a probabili-ty of, .075. This means that there
is less than-etght chances in a hundred thaf this difference could be du.e
to rendomcvariationkWchildren's test tcores.0
6.56.46.36.26.16."0
2 5:9.5.85.75-.6
5.5
'1'23
GA
TE
S-MacG
INIT
IE,
LE
VE
L D
RE
AD
ING
CO
MPR
EH
EN
SION
.
CA
M M
ON
ITO
RIN
G SPA
N
sTi
s'
II
III!.1
2029
SCH
OO
WE
EK
NU
MB
ER
T2I
I
39
MO
NIT
OR
ING
SPAN
WIT
H PR
E-T
EST
AN
D PO
STT
EST
ME
AN
SA
DJU
STE
D T
O A
CC
OU
NT
FOR
PRIM
. DIFFE
RE
NC
ES IN
EX
PER
IME
NT
AL
AN
D4(
NT
RO
L G
RO
UPS IN
BE
ST R
EA
DIN
G197:
CARTS 3 anc."4 show the
in their'performance on
heosion-and Math Compqation.
progress of 2 clsasses throu
the California Achievement I
.
CAM was first introduced in math and.Chart 4 Northeast Elementary
where CAM was-first introduced in reading. Althouh no research design
nor statistical tests were applied in these Bases, very interesting spurts
ades 4, 5 and 6
st - Reading Compre-
Chart 3 is for Twinin'es Elementary where
of-growh are noted for the children participating in the CAM system4
relative to when the system was introduced.
1
117
81
7.5
7.9
6.5
6.0
55
5.0
4.5
r.
-109
READINGCOMPREHENSION 2:1=MATHCOMPUTATION gumwmon.
CLASS-OF1982
CLASS OF1981
BESTMATH
INTRODUCED
BESTREADING
INTRODUCED
MAY'71 '72
MAY FEB' MAY'73
I
FEB MAY'74
TWIN PINES ELEMENTARY SCHOOL'a
a
4
tO
8 11
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
-110:
READINGCOMPREHENSION m=ce
MATHCOMPUTATION CLASS OF:
1982-
LASS OF'1981
BESTREADING
INTRODUCED
BESTMATH
INTRODUCED
MAY71 . '12
MAY FEB MAY 3 FEB MAY'73 74
NORTHEAST ELEMENTARY SCHOOL
119
-111` NI
.
\ART 5 (handout) is a comparison of the mean scores from the sixth grade
NYS PEP Math Test between two groups. One group is comprised of children
in the three'EOtarget schools that first received CAM.: The other group
Is comprised'of the children in the other eleven schools in the District
which-have not used Ab1. uring the time of the comparison. The progress
of theTAM schools .counters the New -York State downward trend in sixth
grade mathematics.scores as reported on page II of the Statewide. Report of
October 1973 Reading and Matbebatics Test Results. The non-CAM schools
mean math scores,are alsofollowing a pattern, similar to that of the state
as a whole. Trends such as these provide substantial evidence that the
CAM system in Brentwood Eis an effective.tool for use in helping children
acquire mathematics skills.
120-
. ..!.+
;1.50
31.25
31.00
30.75
30.50
30.25
30.00
29.75
29.50 1'.+
29.25
29.00
28,75
28.50
28.25
.28.00
27.75
27.50
27.25
27.00
26.75
26.50
,A111111.,106.111... ....11./M11.11.S.411.0
'112 -
BRENTWOOD UNION FREE 'SCHOOLBRENTWOOD, NEW YORK
Non-C.A.M. Schools
.C.A.M. Schools.
FIGURE 1:
Mean Score Comparisons on Three Administrations of the Sixth.Grade N.Y.S. P.E:P.
Math Test of Three E.S.E.A. Target Schools using C.A.M. in Math to the other Eleven.
Schools in BrentWood not wing the C.A.M. system.
10 71 10 72:
12i
4
°
4 .
CONCI.VSMN',
),0 -
0,
,os
An inforMation system that giveS regUlar:rePorts on'achtT
event and
retention:heips.bOth the learner a, nd the'teacher. .The CAM. nformatiOnN
r
system-does not prescribe materials or methods'lmt requires frequent
0
teacher decisions concerning strategy and tactics .to help.' students achieve.
'Brentwood's three year results on.a variety Of,teste support the idea
that specifying tasks and testing Oerformande.will improve the student's
performance in reading and math for grades 4, 5, and 6%
Observing student's behavior on these specified tasks seems More ap-.
proprlate than measuring against a norm which sorts all learners into4
above and below percentiles or stanines.
Cs
0
A
122
0
D'f7
st.
,
-114-
Tbe Product. Evaluation submitted to the Klepak Commission used
norm-referenced tests for comparison. If we really believe In
criterion referenced tests, the analysis of objectives and per-
formance used in evaluating a program should measure student
growth against the original criterion. Mr. David Holt has compiled
the following report on our student's progress toward the set of
objectives for each level. Overwhelmingly there is- positive growth.
Where there is not, the math committee will study the situation to
bring about ore of the following:
1. better student performance
2. change in the Objectives,u.
3. change in test items
-If: change in method or material for instruction.
The evaluation of the program to this point has encouraged
the decision to monitor reading, and mathematics in the.junior high
schools as well, begipning in lt5-76.
123_
10
I
115S
BRENTWOOD UNION FREE SCHOOL DISTRICTBRENTWOOD, NEW YORK
TO: Arthur R..Brieger
FROM: David S. Holt
RE: BEST MATH ANALYSIS OF_OBJECTWE PROGRESS DURING THE 1973-1974SCHOOL YEAR
DATE: April 9, 1975
0
At the request of`Mr. Fournier, I conducted en analysis of the BEST Mathprogress during the 1973-74 school Year.
Attached, are several tables that examine the progress of children In_achieving success on each objective within each math level. This is thefirst ccmprehensive view of math achievement shown in terms4of the
r s
criterion referenced test scores for a full year period. The resu'ts ofthe analysis are very interesting; they show important positive changes;
° they should be very useful to the BEST Math Revision Committee.--
The first table in each level is a frequency distribution of the percentcorrect for each objective on the first test pre - instructional and of thepercent correct for each objective on the last test poii-istructional.This table shows the median percent correct:olor all objectives pre-instry'c-,tional, for all objectives post-instructional and for the,incre*ase rn medianpercent correct; It also Shows the range of percent correct for theobjectives On the, first test and on the last test. It will be noted thatin all three levels the range narrows considerably as the median percentcorrect is raised: These two bits of information show that ti-c pupil'sprogress has the pattern expected when teachers utilize the procedure ofteaching for achievement of behavioral objectiveS and those objectivesare *measured with: criterion referenced test.
.The second table providet data on individual objectives. The first columnltsti in which semester the objective was taught STd the second'column liststhe percent correct of the items for each objective on the firit test prior
to instruction, The next column indicates the percent correct of the itemsfor each objective on the last test of the semester, while the fourth column
° lists the increase in percent correct, i.e., column three minu-slcolumnColumn five lists the perce:it correct of the items for the objectives taught
--both semesters; and the next. column 1:sts the 4ncrease in percent correct,i.e., column five minus column two. The data inE-ctiumn four also have plussigns (4-) beside the increases that are above the median for the group of .
objectives. Those objectives above the median increase-are verbally identi-
fied in the last column.
The last table presents the frequency of increases in percent correct for
all objectives for only one semester and for both semesters.
These data summaries will help the review committee make decisions about
which objectives to keep and which objectives need revision. 1 would be
glad to explain these tables in person if you felt it would be helpful.
DSH:cbcc: Hr. DiPietro
Hr Foureer
124
-116-
BRENTWOOD. UNION tREESCHOOL DISTRICTBRENTWO0D, NEW YORK
GRADE 4 - BEST MATH
Frequency of % RightPre-Instructional
f cf
96-100 .
91 -95
86- 90
76 80 1 1 42
17 75
667 70
61 -.65 11 '2 41.
.56= 80.
51 55 1,141 5 39
46- 50 .1.4i1 111.. 8 34
41- 45 ..1-14-r -t-HI 1 11 26(Median n 43.2)
36- 40 1 1 15
31- 35 1111 4 .14
26- 30 111. 3 10
21- 25 111 3 7
16- 20 1111 4 4
11- 15
.6- 10
S
4Frequency of % Ri.ghtPost - Instructional
111
f cfg,
3'_42
.11 39
3 35
4. 32
-8. 28(Medi 66.1)
5 20
7 15
4. 8.
Difference between medians is 22.9
Range 64 'iange uk 54
125
I
dr
Alb
1;
-117-
BRENTWOOD UNION'FREE SCHOOL DISTRICTBRENTWOOD, NEW YORK
GRADE 4 - BEST MATH
Test IS- Right
Objective Semester Pre -In-
Number Taught structional
4u 1-
413 1
414 2
415 1
416 2
417 2
418 2
421 1
422 1 4
423 1
424 1,2
425 2
426 2
427 2
431 2
432 2
433 2
434 2,
435 2
436- ' 2
437, , % 1,2
451 1,2
Test 6,,,% RightPost-In= ,
structil
Test 12Increase % Right Increase
in % post-In- in % 'Objective
Right structional Right ,-;:Description
46 64 lA
345t
22
43 59 16
-.
43 56 13
25 37 12
20 53 33 +
50 69 19
64 83"- . 19
53 ..17
.
39 76 37 +
19 46 27 + %, ,,- 52 33
49 66\17
41 69 28 +
25 '57- 32 +
26 48 ' 22 ,
35 68 33 +
20 46 26 +
45 86 . 41 +
55 84 . . 29 +
34 48 . . 14
24 55 , 31'+
..j'64 40
43 60 17 71 28
12G
Conmork;FactoA
Multiplicetior
Division
Multiplicatio
Division
EquivalentFractions
Mixed Numbers
Add Fractign1iva= Denom.
Subtract Fracwi= Denom.
EquivalenFractions
t'
. .
Test% Right
Objective Semester Pre-In-Number Tauoht structional
452 1,2 41
-118-
BRENTWOOD UNION FREE SCHODL.DISTRICTBRENTWOOD, NEW YORK
GRAD(4 - BEST MATH
Test 6 Test 12
% Right 'Increase a RightPost-In- in 7; Post-In-
structional Right structional
68 - 27+
461 . 55 69 14.
44 1 , 49 60 11
463 .52. 63, 11
464 1,2 49 ,68 19
465 ' .1,2 43 7.128 +
466 2 31 56 25 +
467. 1 48 61 13.
468 1,2 64 91 27-+
- 469 2 44 64 20.
, 4610 2 27 55' 2&+
471 19 60 41 +
472., 46, 70 g4
473) , 2 50 66. 16
4741 2 42, 69 27 +
475 2 44 70* " '26 +
, 476 2. 29 55 26 +
482 54 79 25 + -
483 76. 90 14
'127
77
73.
81
72
90,
Increasein % ObjectiveRicht Description
36 Basic operations w/numbe:sent
Number sentwiFractions
18
"32
29 bDivisiod word'Problems
+, - word prow/fractions
26 -Reading graph
'Division wordproblems
English linea, measure w/ +
Precition airlin.measureme
.1>
English meant.wiAdd.& Subt,
Time measure
English line'measure w/ -
Identify 3Dfigures
Increase in% Right
56-60.
451-55
46-50
41745
46-40
31-35
26-30
21-25
.16-20
6-10
1-5
St
BRENTWOOD UNION FREE SCHOOL DISTRICTBRENTWOOD, NEW YORK
GRADE 4.- BEST MATH
Objective FreOne Semester
f cf
11 t 2 42
IT 40
1111 4 38
44i/ 441-1 1 11 34
.11-11 5 23
23.5 Median44-11 34-It 10 18
44-11 111 13 8
F
128::
Oenc, Distributionboth Semesters
t f 'cf.
11 2 8
11 2 6
111 3 4
1' 1
3d.5 Median
Or I ,te moo
-120--
BRENTWOOD UNION fREE SCHOOL DISTRICT.BRENTWOD,NEW YORK
.,
GRADE 5 - BEST MATH
Frequency of % RightPre-Instructional
56-100
91-.95
86- 50
81- S5 I 1 58
f cf
-'76- 80 1 1 57
71- 75
66- 70 111 3 56
61- 65 1111 4 53
56- 60 . 111 3 49
i1- 55 1 1 46
46- 50 1.14T 5 45
41- 45 1.141 11 7 40
36- 46 111 3 33
31- 35 111 3 30(Median = 33.8)
26- 30 .14-1-1 14-11- 13 2711
21- 25 .3.4-13- 11 7 14
16- 20 1111 4 7
11- 15 1 3
6- 10 11 2 2
1- 5
'Frequency of % RightPost-Instructional
t f cf.
1 1 54
11 2 57
11 2 55
4417 11 7 53
141 111 8 '46'
-.1- H11111 9 38(Itedian = 65.5)
114/ 114T 11 , 12 29
11-11.-1-1-11 10 17
.7
1 2
Difference between medians is 31.7
Range .75 Range = 59
129
0
Objective'Number
511
513
514
I
51
516
517
518
519
521
522
523
524
531
532
533
534
535
536
. 537
538
1
'1.121- .7
BRENTWOOD UNION FREE SCHOOL DISTRICTBRdTWOOD, NEW YORK
:GRADE 5 - BEST MATH
Test IRight
Semester Pre-In-Taucht structional
1 68
1 60d
1 2542.
2 28
2 30
2 38
2 43
2 61
82 %
1 70"
." 1,2 v 46
1,2 35
2 36
, 2 33
2 22.
2 16
2 17
:2 52
2 26
2 28
Test 6% RightPost-In-structionpl
Increase.in XRight
Test 12% Right
. Post -In-
structional
75 '867 7
65 40+
41 13
51 21 .
63 25.
68 25
75 14
86 4
82 12
76 30+ 660
65 30 + 56
63 27
64 31 +
60. 38 +
55 39 +
58 .41 +
74 22
63 37 +
62 34 +
130
Inereatein S ObjectiveRight Desdripticn
Rounding to1,000
20. Multiply two3-digit nos.
19 Division of5-digits-by 2
Bits w/remainder
Least comrondenominator
Adding fract.w/unlikedenominators
Subtract fracw/unlikedenominators
Multiply uhol.I and a fract
Add mixedw/unlikedenominators
Subtr. mixed
w/unlikedenominators
-122-7
.
BRENTWOOD UNION FREE SCHOOL DISTRICT
BRENTWOOD, NEW TOM'
GRAPE 5 - BEST MATH
\---,
Objective .SenesterNumber Taught
Test I'% -RightPre-In-structienal
st 6 '
. RightPest-In-structional
Increasein % '
Right
Test 12% Right
. Post -In-structtonal
Increasein %Right
539 2 , 39 64 25
5310 2 27 57 30
5311 . 18 57 39 +.
541 2 27 80 53
542 2 42 77.. 35
54319 57 38+
544 2 63 74 11
545 2 . 10 52 42+.
551 1 25 52 27
552 1 28 66 13
553 1 23 63 40.+
554 1 27 72 Q 45 +.
.
555 1,2 48 75 27,
69 21
<:)556 l',2 47 67 20 65 18
557 2 45 74 .29+
558. '2 4919
561 1,2 33 4 11 54 21
562 1 78 89 11
563 1 58 74 16
564 1i2 47 66 19: 70 23
134cv
ObjectiveDoscrintivi
Subtr. mixedf /unlikedenominators
Division offractions
Changing fracto decimals
Changing dec.to fractions .
* Adding decina
Place valuefor decimals
Associativepros. ofmultipli catic
Coriputativepron. ofmultiplicatic
, I
Missing frac,# facts in #
sentences
123.;
...,- f,
BRENTWOOD UNIOn FREE SCHOOL DISTRICTJ SRE4IUD4D,-NEW YORK
GRADE_ 5 - BEST MTH
Test I TeSi 6 Test 12% Right % Right Increase % Right Increase
Objective Seester Pre-In- Post-In- ..in % PoSt-in- in % Objective.
e Nu=ber Taucht structicnal structional Right structional Right Description.
565 1,2 - 43 73 . 20
566 2 22 59 37 +
567 1,2 41 60 19
568 1 63 80 17
569 1,2- . 26 71 '45 +
571 1 27 56 29+
572 1,2 60 83 ,23
64 21,
70 29
60.
34 '
85 25
Word problemw/add.or subtfractions
Averages
English lineameasure /add& subtraction
575 27 60 33 + English liquimeasure w/add& Subtraction
576 . 1 27 61 34 English weig;-measure w/adc
o& subtractior
577 1 30 72- .42.+ ! Tim; measure
'%//addition &subtraction
.
578 1 66 79 13
581 1 -63 92 29 +
582 1,2 P. 62 51 + 64 . 53 Area ofrectangle.
r -583 2 9 36 ' .9 .
585 . I 41 78 37 + Properties 0line segnept
586 1 24 77 53 + Cl ass i ficatit
Findingperimeter
587 1 21 76 58
. 588 2 41 69 28
in
132
of angleS
Measurementof angles
Increase inght
56-60,
. 51-55 T111
46-50
-124
BRENTWOOD UNION FREE SCHOOL DISTRICTBRENTWOOD. NEW YORK
GRADE 5 - BEST MATH
.*
Ob'ective Fre uencv DisfribmtionOne Semester Both Semesters
cf t f cf
41-45 *P 5 54
36-40 4.11-1- 1111 941- --5 40
26-30 14-1 10-Median 27.5
21 -25- 4441 1 6
16-20 4.11t 11 . 7.
11-15 14.4i 111 8
6-10 111 3
1- 5 1 1 1
1 1 10
35 1 1 9
25 44-li 5 823.0 Median
19 111 3 3
12
4
133
-125 -
BRENTWOOD UNiON.FREE SCHOOL DISTRICTBRENTWOOD, KW YORK
GRADE 6 - BEST MATH
frequency oi% RightPre-16itructional
96-100
91- 95
t
I
f cf
86- 90 1 , 1 54
81- 85 1 1 53
76- 80 11 2 52
71- 75
66-. 70 111 3 50
61- 65
56- 60 11 2 47
51- 55 1111' 4 4 45
46- 50 1111 4 41
41- 45 111 3 37
36- 40 1}11- 1. 6 34
31'- 35 1341 1 6 28 .
(Median 34.7)26- 30 1141- 1411 11 12 22
21- 25 144/ 5 10
16- 20 1111 4 5
11- 15 1 1 1
6- 10
1- 5 .
Range 79
6
..134
Frequency of % RightPost-Instructional
f cf
11 54
111 3 53
111 3 50
1111 4 47
4411 1 6 43
1441 111 8 37
1141 1111 9 29(Median it, 64.4)
114T 1441 11 12 20
.1111 4 8
111 3 4
1 1 1
Al0
Difference betwetnmedians is 30
Range - 50
4
SemesterTaught
,BRENTWOOD UNION FREE SCHOOL. DISTRICTBRENTWOOD, NEW YORK
Test I% RightPre-In-structional
611 2 24
613 , 18
614 2 25
-616 2 29
616 1,2 57
6)7 1,2 'i3618 .1
619 .2 .41.
Z21' 1 86
622' 1 -84
6231
68
624 1,2 47
631 1,2. 36
632 1,2 27
633 1,2 . 28
634 1,2 19
635 1,2 30
635 1,2 31
637 1,2 22
GRADE 6 - BEST MATH
Test 6", Right
Post-In-structional
. 59
46-
61
Test. 12
Increase % Right increasein % POst-In- in Objective'Richt -structional ight Description
35+
27
59
79
70
62
30
22
17
27
+
59 °. 18
'11 5
88 4
78 10.
63 16
58 22
59 32 +
69 41 +
61. 42 +
71 41 +
, '57 26
62' . 40 +
13
111
ScientificNotation
79 22.
68 15
62 15
58 22
62 35
63 35
59. 40
65 35
55 24
53 31
Rounding tonearest decimalfraction
Desimal in-equalities,
+ Fractions w/unlike denomina-tors.
- Fractions w/unlike denomina-tors
Multiplicationof Fractions
Division ofFractions
- Fractiont w/unlike denomina-tors
ObjectiveNumber
SemesterTaught
638.
639
1,2
1,2
641 1,2
642 1,2
643 2
644 2
64S 2
646 2
647 2
I, 648 2
651 2
652 2
653 2
654 1
661 -1
4
662 1,2
663
664 1
BRENTWOOD UNION FREE SCHOOL DISTRICT-BRENTWOOD, NEW YORK
GRADE G - BEST PATH
Test I Test 6 Test 12
% Right S Right Increase % Right Increase
Pre-In- Post-In-, in S . Post-In- in % Objective
structional structionaT Right structional Right Description
47
42
20
..'
46
30
29
77
40
28
27
23
29
to
19
, 49
55
28
43
56
75 28 + 72 25
70 28 + 70 28
7? 52 + 66 46
.
79 72 26
58 28 +
. 46 17
84 7
60 20. .
60 32 +
52 25
67 44 +
59 30 +
60 41 +
77 28 +
71 16
'36 8 50 22'I
58 -' 15.
71 15
136
.Hultiolicationof Fractions
Division ofFractions
. '
Place value for,decimals through10,000ths
Reading & writingdecimals through10;030ths
Changing Fractioito decinals
,,.4,. .
Division ofdeCimals throughhundredths
Changing ratioto perrPnt
Finding Percentof natural no.
Finding what %_1!
one no is of
another
Finding averageof whole no.
Objective. SemesterNumber Tamoht
-128-
BRENTWOOD UNION FREE SCHOOL DISTRICTBRENTWOOD, NEW YORK
Test I
.0.
GRADE 6 --BEST MATW,
Test 6 Test 12% Right. % Right Increase % Right IncreaSePre-Ih- Post-In= in'% Post-In- in %
structional structional Richt structional Richt
665 2a . 25 55 30 +
666 1.
76 88 12
667 2 28 69.
yy
41+
668 37 56 19
671 1, 39 69 30+
672 - 1 38 62 24
673 1,2 .35 71 36 +. 77 42
675 26 64 38 + 61 35
676 1 35 63 28 +
0
677 "1 35 67 32'+ .,
. 681 1 68 90 22
682 41) 1 51 81 30+
683 2 . 47 34+
684 1 53 , 82 29 +
.685 1,2: 36 76 do+ 68 '132
.
686 1 67 75 8
687 1,2 35 81 -46 + 71 36
o.
13 1"/4
ObjectiveDescription
Word problemsw/, percent
Proportiorls w/cneunknown in no.sentence
ti
English linearmeasure
Metric"systeminternal conver-sion
LEnglish liquid'measure or -
English weightmeasure w/4- or -
Time measurewki or -
l'lar prism-
Straight line,segment properti
Naming andmeasuring angles
Finding areaof rectangle
Finding surfacearea of rectengu'
Naming parts ofcircles &measuring. arcs.
Increase in% Right
-129-
BRENTWOOD UNION FREE SCHOOL DISTRICTBRENTWOOD, NEW. -YORK
GRADE 6 - BEST MATH
Ob'ective Fre uencv*DistributionOne Semester Both Semesters
fr56-60
51-55 1 1 54
46-50 1 .1 53
41-45 4-1-11 1 6 52
36-40 5 46
31-35 -1-1 6 41
26-30 4+1-1 14 35
111127.64 Median
21-25 44-1-fi 5 21
16-20 4441-11. 7 16
11-15 111 3 9
6-10 1111' 4 6
1- 5 11 2
°138
t
1
1
11
1144
11
11
4,
f cf
1 19
1 18
2 17
1 6 1530.92 Median
2 9
5 7
2
0-
0
a
-130-
X. Budget.
A. Financial Budget
I. Start-up Cast - ltt year
a. 25% per student charge by Grumman
Data System for 1100 students.
ESEA 'funded.
b. $50.00 per team leader for extra
services. '
c. Three principals X 3 weeks salary
for workshop and writing.
d. Three teachers X 3 weeks salary for
workshop and writing'
Total Start-up Casts $10,775.00 ,>
$ 275.00
1500.00
5400.00
3600.00
2. Current Costs
a. 9t per student per test processed by
Brentwood Computer,
b. Twelve Math team leaderseaders at $500.00.
C. Net cast to district for writing
(summer '75) Junior High objectives
in math and rewriting elementary math
and reading. Total Cost is $20,060.00
but through BOCES Cooperative Aid
Project, 80% (district aid ratio) is
reimbursable.
129
4
4 ..131."
d.. Twelve Aides X 3 hdUrs per week.
B. Time Commi tment-
The Curriculum Coordinator has g(men up to 40% of his time in
some periods to the management of th(s project. .
It has involved all teachers and administrators inestimably
from a time point.of view.
4
140
-132-
XI. Self Assessment. Luck bad great deal.-to do with the
successful ithpiementat of thti program. A would ike to believe
that hard work had even more to do with it. I ha r responsible
for the development of educational progrAms in the 'district
since 1963. Many of our programs have achieved national distri-,
Inition. BEST however, was the most complicated, needing more
matefkaTs, people and strategy than any other.
Its scope is overwhelming in some regards. It comes at 1
time when bOavIoral objectives and accountability are contrpver-
sial. It has done what it was supposed to do and yet it will never .
be finished. It cori s to offer great hope for.the organiiation
of more effectiv nstructional programs. It is modern in its ust
of technology an up-to-date management systems. Yet the potential
for mis-use is significAnt:. It Must be watched because if it is
pis-used, at best it would 'fail and disappear; at worst it has
the potential to be used for oppressioll of students and faculty.
This is probably true of all effective measures. Managers are
usually caught in between doing nothing and doing some very pos-6
itive things which have potential for mis -use. ,lust as weare not
likely to give up books, radio, newspapers and television because
they have the potential to betmis-used, we will not pass.up this
fine tool for that same reason. This becomes just one more factor
that 'requires professional vigilance.
Assessing thy own performance in the program, I can't think of
14i.
-133-
much that I would change. If were doing it again, I would
.
like to have more money and a larger steff....
We once had 'a lroup ofAisltors from one ofthe school Atricts
in New York City. (roughly the size of Brentwood) They were ex-
Rending $300,000 .in an attempeto implement this systems The
project coot lnator worked with the project as his one and only job.
With a tiny raction of that budget; with this a small part of, my4
responsibility, theretwere times when it seemed veny-diffitult.
,
Since the program is a success I wouldn't change any decision-
for fear that it would precipitate the development of some uncon-
trollable.situation.
ro
149,
d
. von., 0...0.... .40 1.1.1
-134-
XII. Bibliography".
Cornell, Terry D. "Educational Accountability: A Format forMonitoring the Tebching-Learning Process." EPIC DiversifiedSystems CorporationEducational Innovations Press, Tucson,
1971.
fr
, "'Instr IctionalModule Criterion Referenced Testing," NewYork State Education' Department, Bureau of Research, Albany,
. 1971.
Ellit Allan, B. "Study Guide for Educational Technology andSystems" Nova University.
Gorth, William and O'Reilly, Robert. "How Does CAM Relate toCourse Structure" Instructional Module 3200. SPPED, New YorkState Education Department. 1972.
Gorth,. William and Allen, Richard and O'Reilly, Robert. "Instruc-tional Module 3000" SPPD, New York State Education Department1972.
(xorth, William, O'Reilly, Robert and Pinsky, Paul. "ComprehensiveAchievement Monitoring". Center for Educational Research,University of Massachusetts. Amherts, 1974.
41.
Jencks, Cristopher, et al. "Inequality, A Reassessment of theEffect of Family and Schooling in America." Basic Books, Inc.Pub., New York, 1972.
Krathwohl, David and Bloom, Benjamin and dasia, Bertram B. "Taxonomy" of Educational Objectives." David McKay Co,., New York 1964.
Mager, R. F. "Preparing Instructional Objectives" San FranciscoFearon Publishing Company, 1962.
Popham, W. J. (Ed) "Criterion Referenced Measurement" Educational
Technology Press, Englewood Qiffs, N.J. 1971.
Rubin, lou)s,J. "Curriculum and Instruction" Nova Study Guide, NovaUniversity, 1972.
Skinner, B. F. "The Technology of Teaching" New York Appleton CenturyCrafts, 1968.
Stufflebeam, Daniel. "The Process of, Evaluation An Easy,.Example"Unpublished handout distributed at Cluster. meeting An Education.
a
143
XIII. Appendix.
A.
-135-
MATH
Printout from Computer
B. Manual for Parents and Students
C. Visitors Guide
D. Mastery Tests
E. Cross Reference Guidei
F. Answer Keys
G. Objectives Printout (reading)
H. Math Tests all levels
I. Student. Update Printout List.
J. Practice Book for Level 6 - Idward Harris
READING
Manual for Parents and Students
L. Cross Reference Guides
M. Objective Bank.
N. Test Item Bank
0. Reading Tests all - levels
P. Answer Sheet
Q. Reading Manual for Visitors
R. Reading Mastery Tests /
S. Mastery Answer Key