National assessment: how to make it betterDylan WiliamKing’s College London

...the model that says ‘learn while you are at school the skills that you will apply during your lifetime’ is no longer tenable. These skills will be obsolete by the time you get into the workplace and need them, except for one skill – the skill of being able to learn. It is the skill of being able, not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they are faced with situations for which they were not specifically prepared. (Papert, 1998)

What assessments do we need?

There is only one 21st century skill

The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good school-master (sic) is known by the number of valuable subjects which he declines to teach.

Sir Richard Livingstone, President of Corpus Christi College, Oxford, 1941

Who are assessments for?

Employers/FE/HE Parents Students and teachers

What do employers need?

Examination results are used:To gain information about students’ skills competences attitudes ‘trainability’

(criterion-referenced uses)

To sort and rank applicants(norm-referenced uses)

Why are employers unhappy?

Things are getting better:average IQ has increased (Flynn effect)school achievement has increased

But:needs of work have increased morelink between IQ and exam results is weakening(can’t use exam results as proxies for ‘intelligence’)

teaching to the test has narrowed the curriculum(can’t generalise to things that weren’t tested)

Time

Scores

Lake Wobegon

All the women are strong, all the men are good-looking, and all the children are above average

X

Improvements are limited

Scores on national curriculum tests in mathematics and English for 11-year olds are increasing

Scores on other, comparable, tests have remained constant

So, improvement is limited to those things that are actually tested

Improvements are transient

Proportion of 11 year olds achieving level 4 in mathematics has increased steadily over the last five years

According to Ofsted, 25% of those who achieved level 4 at the end of year 6 fail to achieve the same level at the end of year 7.

So, achievement has improved at age 11, but not at age 12.

Goodhart’s law

Every performance indicator loses its usefulness when used as an object of policyRailtrackNational Health ServiceTest and exam results

The clearer you are about what you want, the more likely you are to get it, but the less likely it is to mean anything

What do parents need?

Before choosing a school:“How well will my child do if they go to this school?”

Failing that“How well did children similar to my child do when they went to this school?” (value-added approach)

Failing that“How well do the children who go to this school do?” (raw results approach)

Once a child is at school:“How is my child doing?”

What do students & teachers need?

Students need to know:where they are in their learningwhere they are goinghow to get there

Teachers need to knowwhere students are in their learning

what to do about it When assessment supports all these, it is formative

Formative assessment

Frequent feedback is not necessarily formative

Feedback that causes improvement is not necessarily formative

Assessment is formative only if the information fed back to the learner is used by the learner in making improvements

To be formative, assessment must include a recipe for future action

Formative and summative

Fine-scaled data that supports formative uses can be aggregated to serve a summative function

Aggregate summative data cannot be dis-aggregated to identify learning needs

Assessment for formative purposes should be the foundation of all assessment in schools

The problem

Our current assessment system provides poor answers to the wrong questions, and directs attention on the assessment of learning, rather than assessment for learning

Three main problemsReliability (too many wrong levels/grades)

Validity (misleading levels/grades)Focus (summative rather than formative)

National Assessment

Should provide parents with meaningful, accurate data on school performance

Should reflect all the desired outcomes of schooling

Should yield information which is comparable across schools

Should not compromise the school’s use of assessment data for formative purposes

What can be done?

Retain tests and examinations Supplement these with other formal (externally marked) assessments:Extended course-work tasksGroup tasksOral and practical assessments

Students assigned randomly to different assessments

Results provide an ‘envelope’ of levels or grades that the school is allowed to award to students (up to the school to propose how).

Would this work?

It would be better if only value-added scores were published

And even better if no scores were published

But, even with ‘league tables’levels and grades would be based on more than one or two short tests/exams (increased reliability)

the only way to increase levels or grades would be to improve the achievement of all the students on all the tasks, tests & exams (increased validity)