Short reports of thelatest research inMedical Education

A test of progressJane McHarg, Paul Bradley, Suzanne

Chamberlain, Chris Ricketts, Judy Searle,

John C McLachlan. Assessment of

progress tests. Medical Education 2005:

39(2); 221–227.

Recommendations on progresstest marking should make theresults more meaningful forteachers and students.

Researchers advise medicalschools using progress tests as ameans of assessment to use:

� Norm referencing rather thancriterion referencing

� Negative marking rather thannumber-right marking

� A discontinuous rather thancontinuous scale.

In addition, they suggest thatgrades should be weighted to

Digest

118 THE CLINICAL TEACHER December 2005 | Volume 2 | No 2| www.theclinicalteacher.com

favour the most recent outcomesand that grade boundaries shouldtake into account rules on reme-diation and progression.

Progress tests, invented by theUniversity of Missouri-Kansas CitySchool of Medicine and the Uni-versity of Limburg in Maas-tricht1,2,3, emerged alongside newcurricula methods such as prob-lem-based learning or self-direc-ted learning that encourage ‘deeplearning’, rather than learning byrote.

The whole medical schoolsits the same test together andthe knowledge growth of eachyear group and each individual ismapped throughout the course.The questions are drawnrandomly from a large bank ofobjective multiple choice or true/

false questions which test thecomplete domain of knowledgea student should have ongraduation.

Progress tests break the linkbetween learning and revision1

because students cannot ‘cram’for a single module but instead

have to acquire informationcontinuously in such a way thatit will be available whenrequired. But first year stu-dents – used to doing well atschool – can become demoral-ised at receiving low scores intheir early tests.

The study recommends med-ical schools conducting progresstests ‘should be aware of thisproblem and manage expectationamong students’.

The authors believe norm ref-erencing – comparing students’abilities with those of their peers– is preferable to criterion refer-encing, which assesses perform-ance against fixed criteria.Despite the fear that, howevergood a student cohort, some willalways be assessed as unsatisfac-tory under norm referencing, theauthors argue that students rarelyunderperform in more than onetest. If a student consistentlydoes badly on progress tests, thiscould suggest he or she is failingoverall.

The study’s advice on negativemarking aims to overcome theproblem of students scoring wellin a multiple choice test simplyusing guesswork. If getting awrong answer means points arededucted from their score, stu-dents are less inclined to guess.The trick is to weight the wronganswers in a way that encouragesinformed deduction, but deterswild guesswork. Including a ‘don’t

first yearstudents canbecomedemoralised

December 2005 | Volume 2 | No 2| www.theclinicalteacher.com THE CLINICAL TEACHER 119

know’ with a neutral score mayalso be appropriate.

A student’s ‘running grade’ of,say, satisfactory, doubtful andunsatisfactory should be calcula-ted on the basis of their last fewtests – although no optimumnumber of tests is recommended.Among the reasons for this, theauthors say: ‘It is important forformative purposes to represent ameasure of a student’s currentability rather than a reflection ofhis or her past abilities.’

The authors have also deviseda way of modelling studentprogression using a first-order

Markov chain – where the nextvalue depends on the precedingone – and a transition probabilitymatrix.

This, they say, will allow amedical school to predict theoutcome of its students – eitherat an individual or cohort level.The same system could be used toinvestigate the consequences ofvarying the grade boundaries anddifferent rules of progression.

The authors say: ‘By combi-ning these results the optimumevidence should be made avail-able on which to make decisions

regarding and individual student’scompetence.’

REFERENCES

1. van der Vleuten CPM, Verwijnen GM,

Wijnen HFW. Fifteen years of experi-

ence with progress testing in a

problem based learning curriculum.

Med Teacher 1996;18:103–9.

2. Van Heeson PAW, Verwijnen GM. Does

problem based learning provide other

knowledge? In: Bender W, Hiemstra

RJ, Scherpbier AJJA, Zwierstra RP

eds. Teaching and Assessing Clinical

Competence. Groningen, The Nether-

lands: Boek Werk, 1990:446–51.

3. Arnold L and Willoughby TL. The

quarterly profile examination. Acad

Med 1990;65:515–6.

Lessons in learningLinda H Pololi, Richard M Frankel.

Humanising medical education through

faculty development: linking self-aware-

ness and teaching skills. Medical Educa-

tion 2005: 39(2); 154–162.

One medical school is reapingthe rewards of a new approach toteacher development, combiningtraditional and non-traditionalelements of education.

Medical academics who havecompleted the year-long Leader-ship in Teaching (LT) coursereported increased self awarenessand renewed energy and enthusi-asm for teaching as well as themastery of new skills.

And, importantly, they havebeen choosing to replicate theirown enhanced learning experien-ces with their students.

The LT course was designedagainst a background of growingdissatisfaction in US academichealth centres. Medical academ-ics were facing increasing pres-sure to give up teaching andscholarship time – the veryfactors that drew them toacademic medicine in the first

place - in favour of clinical workthat increased their centre’sincome.

Its creators wanted to bridgethe gap between teaching skillsdevelopment and self awarenessdevelopment which tend to beviewed as separate learningschemes and combine them into alearner-centred approach that re-mains relatively rare in medicaleducation. In such an approach,learners identify their own learn-ing needs and objectives, plantheir learning activities and as-sess their own results.

The course designers explainthat self awareness - insight intohow one’s own life experiencesand emotional make-up affectinteractions with others – isvaluable for doctors and teachersin delivering effective patientcare and meeting the needs oflearners1,2 while communitybuilding – relationship formation– is key to establishing thera-peutic relationships betweenphysicians and patients.3

The programme focused onareas including small groupdynamics and challenges, giving

No optimumnumber of testsis recommended

120 THE CLINICAL TEACHER December 2005 | Volume 2 | No 2| www.theclinicalteacher.com