This article was downloaded by: [University of Birmingham]On: 15 November 2014, At: 00:35Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Innovations in Education & TrainingInternationalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/riie19

An Approach to Diagnostic Testing in theScottish O Grade Biology CourseHugh Perfect a & Jannet Robinson ba Director, In‐Service, Research and Development , Moray HouseCollege of Education , Edinburghb Principal Teacher of Biology , Deans Community High School ,LivingstonPublished online: 02 Dec 2011.

To cite this article: Hugh Perfect & Jannet Robinson (1983) An Approach to Diagnostic Testing in theScottish O Grade Biology Course, Innovations in Education & Training International, 20:1, 27-35, DOI:10.1080/0033039830200105

To link to this article: http://dx.doi.org/10.1080/0033039830200105

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis, ouragents, and our licensors make no representations or warranties whatsoever as to theaccuracy, completeness, or suitability for any purpose of the Content. Any opinions andviews expressed in this publication are the opinions and views of the authors, and arenot the views of or endorsed by Taylor & Francis. The accuracy of the Content should notbe relied upon and should be independently verified with primary sources of information.Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands,costs, expenses, damages, and other liabilities whatsoever or howsoever caused arisingdirectly or indirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

DIAGNOSTIC ASSESSMENT

An Approach to Diagnostic Testing in theScottish O Grade Biology Course

Hugh Perfect, Director, In-Service, Research and Development, Moray House College of Education, EdinburghJannet Robinson, Principal Teacher of Biology, Deans Community High School, Livingston

Abstract: There is a growing interest among teachers andresearchers in adopting aspects of criterion-referencedtesting in schools, particularly for the purpose of diag-nostic assessment. This article outlines the strategiesdeveloped by the authors for such a diagnostic testingapproach within the context of the Scottish O gradebiology course. Also described is the use made of a

microcomputer-based marking and reporting system usinga software package called SCRIBE. This package enablesthe class teacher to obtain prompt feedback on theattainment of individual students, as well as informationessential for the effective evaluation of the teaching andtesting procedures used.

IntroductionDeans is a large community high school havinga school roll of 1100 in the academic session1982/83. Many of the subjects making up thecurriculum of the school operate a regime ofinternal certification. These certificates providea system of feedback to students on how theyare progressing in a particular subject, and alsoact as a record of what each student should knowor be able to do at that particular stage of thecourse. Certificates gained are kept in a folderwhich serves as a personal profile and provides adetailed record of a student's achievementsthroughout his time at school, and thus may be ofparticular interest to prospective employers.Students following the O grade biology coursecan gain up to five such certificates, commencingat level one and ending at level five. The awardof each level of certificate is dependent upon thepercentage of marks obtained in a series of end-of-units tests sat during the two year course. Theattaining of the level four and five certificatesgives a reasonable indication that a student couldgain a pass in the Scottish Examination Board(SEB) O grade biology papers. At all stagesstudents are interested to hear their test resultsand frequently ask, 'Have I got another certificate?'or, 'When will I get my next certificate?' Thus thecertificates also appear to fulfil a further functionof helping student motivation.

There are some problems associated with thispresent system of certification. For example, anycertificate awarded will only detail those broadobjectives which a student luill probably haveachieved rather than a more specific statement ofwhat an individual student has actually attained atthat particular time. Consequently the authors

have attempted to improve the scheme so that itgives more diagnostic information on the achieve-ment of each student at the completion of a unitof work.

Such a diagnostic testing approach was alsoconsidered to be an important source of feedbackfor evaluating the individualized learning unitsbeing written and trialled at Deans. To date foursuch units have been written based on the medium-term model1 and constructed on a 'core + ex-tension' pattern, with the O grade studentsbeginning and completing the units together.

Besides this advantage of diagnostic testing inproviding feedback on the efficacy of the individ-ualized learning approach, the authors alsoconsidered it would be valuable in:

(a) providing the teacher with specific feedbackon the achievements of individual studentsof the stated objectives of each unit;

(b) providing the students with similar feedbackon their success or otherwise at achieving theobjectives of the unit;

(c) assisting communication between the school,parents and employers enabling a clear state-ment to be made of the individual achieve-ments of each student.

Such a diagnostic testing approach should enablethe particular area of weakness of every studentto be identified, permitting the students, withthe help of the teacher, to undertake any 'remedialwork' that may be necessary.

The following procedures were adopted for theconstruction and use of the diagnostic tests usedwith the 'photosynthesis' and 'water and plants'individualized learning units.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

28 PLET, Vol. 20, No. 1 - February 1983

Devising a diagnostic test

The aim of each of the diagnostic tests devisedwas to investigate the attainment of the students.The tests did not set out to compare the achieve-ments of individual students or to discriminatebetween, say, the more or less able students in aclass.2 The achievement of the students wasassessed in terms of their attainment of theobjectives of each particular unit of work ofthe course.

Stage A: specifying objectives

A detailed specification of the objectives, which itwas hoped each unit of work would achieve, wasrequired in order to facilitate the construction ofa valid diagnostic test. Such objectives had alreadybeen drawn up for the individualized learning unitsbeing tried out at Deans, there being no statementof specific objectives for the present SEB O gradebiology course. The objectives for each unit wereclassified into four broad categories:

(a)(b)(c)

KnowledgeEnquiry skillsPractical skills

(d) Attitudes.Because of the constraints imposed by the presentO grade biology examination, few objectivescould be included in categories (b) - (d). Objectiveswere stated in as unambiguous terms as possible,use being made of clear operational words (state,define, label, devise, describe . . .) together witha specification of what was intended to be an'acceptable' performance for each objective.

The final list for each unit constituted the coreobjectives as required for the O grade biologycourse. Further extension objectives were alsoincluded for the more able students. Many of theobjectives identified in each unit were specific to aparticular unit, whereas others would be developedonly by activities associated with a number ofunits. This latter type included process skills suchas devising hypotheses, devising experiments, orhandling data.3 Students following each unitwere given an adapted 'checklist' of the objectivesto help them with their later revision.

To facilitate test construction and assist communi-cation to the students, the original 24 specificobjectives for the 'photosynthesis' unit wereclassified into 10 broad general objectives.

For example, the following specific objectiveswere all concerned with carbohydrates and photo-synthesis:

Students should be able to:

1. state that sugars and starch are the main carbo-

hydrate products of photosynthesis2. give one example of a plant storing starch and

one example of a plant storing sugar in theirleaves

3. state that sugars and starch are interconvertiblein leaves

4. state that the primary translocatory metaboliteis sucrose

5. state three possible fates for the carbohydratesproduced during photosynthesis

6. state that light energy is converted intochemical energy during photosynthesis.

These six specific objectives were classified intothe general objective:

Students should know the carbohydrates producedby photosynthesis, their interconversion andpossible fates.

The authors have found that the production of afew such general objectives for each unit helps thereporting of the test results to the students, whilstalso keeping the overall number of such objectivesto a predicted 150-200 for the O grade biologycourse as a whole. When required for say revision,the students can convert these general objectivesinto the original specific ones using the unit'checklists'.

Stage B: test construction

The general objectives formed the basis of thetest specification, with items being matched toeach objective. A number of items was requiredfor each objective, since:

1. no single item can test reliably any givenobjective;

2. the smaller the number of items used thegreater will be the error arising from studentsguessing;

3. a single item may generate its own 'noise' interms of its reliability, language used andso on.

The authors aimed for about five items per generalobjective, which for a diagnostic test of eightobjectives would require a 40 item test. Themajority of items were based on the multiple-choice format. Each item was carefully matchedto the appropriate objective to ensure that itmeasured what it set out to measure, ie it wasvalid.

Presumably, in an ideal world a diagnostic itemwould be answered correctly by all the students:the objectives will have been clearly specified;the objective matched to the ability of thestudents; the test item will be valid and reliable;the teaching strategies appropriate; and studentmotivation excellent. However, in the real world

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

Diagnostic Testing in Scottish O Grade Biology 29

the response of students to such an item willvary for a variety of reasons including:

— objective not clearly defined— objective not matched to student's ability— teaching strategies not wholly appropriate— content of item or course not appropriate— items not wholly valid— student motivation poor— student has not developed specific ability

required

There is an inbuilt unreliability into any item andit is unlikely that an item could be expected to beanswered correctly by all students, ie 100 percentof students showing mastery of the appropriateobjective.

Since the questions used in the diagnostic test forthe photosynthesis unit were untried, an arbitrary85 per cent mastery level was given to each itemuntil more specific data permitted a clearer desig-nation. It might also be the case that certain of thegeneral objectives identified for the photosynthesisunit are more important than others and thereforemight ultimately require a higher specified masterylevel.

Stage C: setting the test

A common pattern for setting tests is shown asfollows:

Run throughof test resultswith class:individual helpfor some students

Teachingsequence ofunit A

TEST

Teachingsequence ofunit B

Such a paltern mighl be appropriate if a test hasbeen designed solely for grading purposes, butwould 1>P inappropriate in a diagnostic testingsystem, since it does not permit the teacher todevelop an individualized programme of remedialwork for each student requiring surh help.

Teachingsequence ofunit A

Diagnostictest

Teachingsequence ofunit B

Reporting

Studentscarry outindividualremedial/extensionwork

Consequently, a icarhing/diagnoslicmight consist of:

test pattern

One obvious problem associated with thisapproach is the limited time available for individualreporting/remedial work within the present O gradebiology course. Too much time devoted to theseactivities might impair the chances of somestudents if they were subsequently unable tofully cover the content of the present demandingsyllabus.

The approach adopted at Deans has been toproduce a number of 'remedial' sheets which canbe done by students largely as homework sheets.For each of the general objectives identified aspecific 'remedial' sheet was drawn up. Appendix 1contains an example of one of these. Whilst theauthors realize that the above approach is acompromise it is important to recognize thatdiagnostic testing can be used as a part of thelearning process. The information from such testsused, for example, as a guide for structured home-work, can assist students to be more responsiblefor their own subsequent learning.

Stage D: reporting

Any form of testing is time-consuming, both interms of setting and marking the test, and ofreporting and recording student results. Often thereporting consists only of giving the students anindividual grade and/or of giving a brief reviewof the test to the whole class. Such procedureswould not utilize the information available froma diagnostic test, and students would be unclearabout their own individual strengths and weak-nesses. Similarly, teachers lose the opportunityof organizing remedial or extension work tailoredto the needs of each student.

It has to be recognized, however, that effectiveindividualized reporting can be very demanding ofboth teacher and student time. To overcome thisaspect of reporting the authors were able to makeuse of a microcomputer-based reporting schemecalled SCRIBE, developed at Moray House

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

30 PLET, Vol. 20, No. 1 - February 1983

College.5 SCRIBE is a computer package, the aimof which is to 'provide as much relevant inform-ation as possible from test scores, about pupils'strengths and weaknesses while involving teachersin a minimum of effort'.6 The program enablesthe teacher to use any suitable test and thenprepares a variety of reports from which the mostappropriate can be selected. The data required forSCRIBE consist of: the student names for eachclass sitting a particular test; details of the testincluding, for diagnostic tests, the generalobjectives being tested and the items matched tothese; and the student answers to the test, mostefficiently entered via mark-sense cards completedby the students and then read by an optical cardreader. The system is based on an Apple II micro-computer with 64k memory with the programswritten in PASCAL.

Selected reports are produced on a printer: acomplete set of individual student reports for awhole class can be available within half an hourof the student responses being entered.7

An example of the student report format used bythe authors is reproduced in figure 1.

Such SCRIBE reports permit immediate individ-ual feedback to every student giving details of his/her attainment of the general objectives of theunit. Objectives not attained are reported with anadjacent (*),8 this notation assisting the teacherin recognizing the specific learning difficultiesof each student. In the context of the photo-synthesis unit a number of XTRA sheets wereprepared and matched to each of the generalobjectives. These suggest to the student furtherwork which can be carried out either in class timeor at home (see appendix 1).

The SCRIBE reports also provide a basis for theproduction of individual certificates which nowlist the objectives actually achieved by a student.Such certificates are produced using Deans' ownApple II-based system.

An example of this type of attainment certificateis shown in figure 2.

The production and distribution of certificatessuch as these raise a number of crucial educationalissues. These include:

— how detailed should the information presentedon such certificates be?

— for whom are the certificates intended:students, parents, employers?

— can such attainment certificates be used as abasis for end-of-course summative reports orprofiles?

Such questions are part of continuing educationaldebate and the subject of on-going research anddevelopment.

Stage E: evaluation

The computerization of student records has thefurther advantage of providing the teacher withvaluable feedback on the effectiveness of theteaching programme and on the validity andquality of the diagnostic tests themselves. Tofurther assist the teacher in the evaluation process,the SCRIBE package also produces cumulativestatistics for each test used and for the individ-ual items making up the test.

SCRIBE-generated statistics from 14 classes inthree schools which have used the diagnostic testwritten for the Deans' photosynthesis unit arereproduced in figure 3. This particular data isvaluable in checking the reliability of the individ-ual test items. For example, where the localfacility value is in the region of 70 or over andwhere the item appeared to cause students nointerpretation problems, such items were con-sidered acceptable for future use, as in:

Question 3: local facility value = 81 (that is,number getting item correct in this sample)Which of the following word equations bestsummarizes photosynthesis?

A. sugar + oxygen—» water + carbon dioxideB. water + oxygen—-starch + carbon dioxideC. water + carbon dioxide — sugar + oxygenD. oxygen + carbon dioxide—water + starch

and

Question 9: local facility value = 83During photosynthesis chlorophyll:

A. acts like an enzymeB. absorbs light energyC. acts like a catalystD. reflects light energy.

Where the local facility value falls well below 70, anumber of factors could be responsible and theauthors recognize the difficulty of identifying anyspecific causative factor — whether the item is atfault, or inappropriate strategies used, or thedifficulty of the concept involved. For example,students in all three contributing schools foundproblems with:

Question 7: local facility value = 51Which of the following does not happen to thesugar produced during photosynthesis?

A. transported to root and shoot tips for growthB. stored as starchC. broken down during respirationD. transported in the xylem ~

Similarly, students from all three schools haddifficulty with question 35. This required thestudents to describe a stated experiment and wasmarked on a 4, 3, 2, 1, 0 scheme. Despite the

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

Diagnostic Testing in Scottish O Grade Biology 31

Figure 1. Student report formal

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

32 PLET, Vol. 20, No. 1 - February 1983

DEANS COMMUNITY HIGH SCHOOL

BIOLOGY DEPARTMENT

CERTIFICATE I : PHOTOSYNTHESIS

This is to certify that

has achieved the following objectives, and is therefore able to:

Describe and interprete food chains and food webs.

State that carbohydrates are produced during photosynthesis, usedand interconverted.

State that photosynthesising green plants produce oxygen.

State that light, carbon dioxide, water and chlorophyll are neededfor photosynthesis.

State the overall word equation for photosynthesis.

Describe how chloroplasts are distributed in a green leaf.

State five ways in which green leaves are adapted for carrying outphotosynthesis.

Describe how to test a green leaf for starch.

Describe controlled experiments to show that light, carbon dioxideand chlorophyll are necessary for photosynthesis, and that oxygenis given off.

Construct and read line graphs.

Signed: Teacher Date:

Signed: Head of Lower/Middle/Upper School

Signed: HEAD

Figure 2. Attainment certificate

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

Diagnostic Testing in Scottish O Grade Biology 33

Figure 3. Question statistics for 14 classes for photosynthesis

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

34 PLET, Vol. 20, No. 1 - February 1983

KNOW CARBOHYDRATE PRODUCED XTRA B

FROM CHECKLIST

You should be able to state that :

l i gh t energy is converted into chemical energy during photosynthesis

sugar and starch are the main carbohydrate products of photosynthesis

sugars and starch are interconverted in leaves

the 3 possible fates fo r the carbohydrates produced by photosynthesis.

Write down the fol lowing and f i l l i n the answers (you may use the

'Photosynthesis' un i t to help you):

1. The carbohydrates produced by plants such as i r i s and spider

plants during photosynthesis are: .

2. Geranium produces the carbohydrate during

photosynthesis.

3. The. form of energy used by green plants to drive photosynthesis

i s : .

4. The form of energy stored in carbohydrates such as glucose and

starch i s :

5. The three fates for the carbohydrates produced during photosyn-

thesis are:

(a) .

(b)

(c)

Read the section 'Photosynthesis in a palisade cell' on

page 51 of Mackean's 'Introduction to Biology* .

Appendix 1.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14

Diagnostic Testing in Scottish O Grade Biology 35

practical nature of the photosynthesis unit, itwas evident that students require further practiceand reinforcement in answering this particulartype of O grade biology question.

It should be noted that, since the questions forthe photosynthesis diagnostic test were writtenespecially for this work there were no statisticaldata on them prior to their use.

Conclusions

The procedure adopted by the authors for adiagnostic approach to testing in O grade biologyhas proved both effective and efficient. TheO grade syllabus and associated past questionpapers provide a suitable framework for drawingup both specific and general objectives required.Whilst the preparation of diagnostic tests is time-consuming it is certainly feasible for individualteachers to attempt, but would obviously be moreappropriately done at regional or national level.The SCRIBE program provides a clear and valuablecomputerized reporting system which removesfrom the teacher much menial marking andcollating of reports. The reports produced areimportant in permitting the teacher to developindividual programmes of remedial work matchedto the attainment of each student. The individual-ized learning approach developed at Deans providesa flexible enough framework for using suchremedial or extension work. It may prove possibleto ensure that all students develop those keyobjectives which will be used in later course units,although time and motivation are the two majorconstraints to be considered.

References

1. Perfect, H.E. and Robinson, J. (1982) Anindividualized learning approach to O gradebiology. In The Journal 22. Scottish Curricu-lum Development Service (Dundee Centre).

2. Popham defines a criterion-referenced test asone which '. . . is used to find an individualstudent's status with respect to a well definedbehavioural domain'. Refer to: Popham, W.J.(1978) Criterion-Referenced Measurement.

Prentice-Hall, New Jersey.

3. Whilst it is easy to state that the drawing up ofobjectives is a prerequisite for successful diag-nostic testing, there are fundamental problemsabout ensuring the validity of any such speci-fication. One view would be that the objectivesshould be selected in order to reflect whatcan be reasonably expected of students of aspecified age/ability in a particular subjectarea. Such a list of objectives would obviouslybe dependent on the teacher's views (andothers) of what students can achieve in the timemade available in the school curriculum. For adiscussion of this and related matters refer to:Brown, S (1980) What do they know? A reviewof criterion-referenced assessment. SED.

4. Refer to the discussion in: Black, H. andDockrell, W. (1980) Diagnostic Assessment inSecondary Schools: A Teacher's Handbook.SCRE.

5. SCRIBE has been developed within the SEDfunded S1/S2 Science School-Based Assessmentusing Item Banking project. The directors ofthe project are P.J. Barker (Moray HouseCollege) and S.H. Kellington (Glasgow Collegeof Technology). The research officer is A.C.Mitchell (based at Glasgow College of Tech-nology), from whom further details of theproject can be obtained. The program waswritten by David McArthur, at Moray HouseCollege.

6. Refer to Information Paper No. 3 from theS1/S2 Science School-Based Assessment usingItem Banking project, A.C. Mitchell and S.H.Kellington.

7. For further information refer to: Mitchell,A.C. (1982) Using Microcomputers to helpteachers to develop their assessment procedures:a development project report. ProgrammedLearning and Educational Technology 19, 3.

8. Asterisks are printed when a student's score forthe items of a particular objective fall below theaccepted mastery score.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

0:35

15

Nov

embe

r 20

14