A new look at standards:

What should we want from GCSEs and A levels?

Presentation by Tim Oates

Group Director

Assessment Research and Development (ARD)

13 October 2011

Complex systems

• Our partial data refer to epiphenomena; we need to apprehend deep structures and concealed trends

• We need to understand examinations in the context of ‘curriculum coherence’

• Careful thinking or split thinking? We need to understand the limits of our sphere of action (modesty and self-control) but simultaneously the extent of impact and dependencies (sophistication in understanding and in strategic planning)

Exam system has grown into a monster

• ‘…”the volume of assessment has left awarding bodies overstretched, resulting in errors in questions, high dissatisfaction with marking and inconsistency in grades”…’

• ‘…”the volume of assessment over the final three years of secondary school … requires, if it is to be conducted as well as it possibly can be, a far greater application of resource”…’

• ‘…the OCR exam board awarded an A* to 19 per cent of HMC pupils for Physics, while for Edexcel the figure was almost 25 per cent. For Economics, AQA awarded and A* to 16 per cent of HMC candidates but for OCR the figure was 11 per cent…’.

• Important locus of control issues embedded in this statement

• Important implied and explicit models regarding standards

Times 07 Sept 2011Kenneth Durham University College School Hampstead/HMC

The Cambridge Standards Debate

Have we moved on?

Why the debate?

• The existence of mechanisms for possible subtle grade drift

• Evidence from diverse sources – enough to stimulate anxiety

Promoting clarity• standards of demand• content standards• standards of attainment

• standards over time• standards between specifications• standards between subjects • standards between types of qualifications• teaching standards and standards of education

Mechanisms

1. A period of constant change in structure and content 2. Removal of bias, enhancement of accessibility and

transparency of assessments3. Benefit of the doubt4. Increased support via mark schemes, text books5. Emphasis on inclusion 6. Focus on attainment – solely on ‘exam performance’7. Modularisation and complex routes8. Changing cohorts, subject choice9. Massive investment in education

Where to, from here? • We should understand, with precision, the impact of

efforts to improve standards of attainment• Reduce the frequency and scope of change in

qualifications • Qualifications should be owned through partnership

between schools, Higher Education, employers and awarding bodies

• Fitness for purpose, clarity of purpose and validity – pre-eminent concerns for determining standards

The purposes and functions of qualifications1. Documenting outcomes of learning

2. Supporting specific, valid inference(s) regarding the candidates’ possession of specific knowledge, skills and/or understanding

3. Signalling

4. Valuation of different aspects of knowledge, skills and understanding

5. Discrimination and selection

6. Controlling flows into specific occupations and regulating the labour market

7. Empowering citizens

8. Realigning the control of professions

9. Measuring the level of skills and knowledge in the national, sectoral system

10. Measuring the performance of the education system

11. Ensuring linkage of content of programmes (training) to work

12. Recognising the actual knowledge, skills and competences required in performance

13. Fill gaps and update requirements in knowledge, skills and competences

14. Guaranteeing the quality of provision for learners

15. Guaranteeing the quality of provision for funding agencies

16. Affecting the identity of learners

17. Effecting social integration

18. Conferring status on qualified individuals

19. Processes of recognition, accreditation and ‘valuation’ of prior learning

20. Managing competences within enterprises

21. Offering inclusion

22. Providing orientation, guidance

23. Controlling the education and training system generally and the qualifications system specifically

24. Enacting reform in education and training

25. Influencing the content of learning programmes

26. Conditioning or shaping pedagogy

27. Conditioning or shaping assessment

28. Developing zones of mutual trust (between users of qualifications)

29. Providing an accountability mechanism

30. Invoking specific models of competence

31. Giving status to institutional/provider offering

32. Shifting control from one agency to another

33. Shifting control for assessment in the system

34. Protecting the content, standing and identity of a profession

35. Providing feedback to learners (formative and diagnostic function)

36. Helping teachers and trainers understand the strengths and weaknesses of their provision (evaluative function)

37. Introducing innovation

38. Protecting consumers

39. Delivering public benefits

40. Meeting international licensing requirements

A thousand tiny steps…

• Access, defensibility, and efficiency (drift in item forms)

• Standards maintenance – the magnitude of ‘movement’

(drift in attainment standards)

• Content standards/validity (drift in validity, currency, credibility and learning

progression)

The interaction of public understanding and policy

• The shock and horror of ‘error’

• ‘Measurement accuracy’ – monitoring it, reducing it

• Dylan Wiliam: ‘….80 hour exams should do it…’

• How long would you like assessment to be, what do you want them to look like, what do you want them to cost?

• Instruction-sensitive assessment

A/AS level examinationsCandidates Entries Papers Cost

(includes correction for inflation)

Expenditure on education in England

1970 103,000

(£89.32)

441,898

(£20.82)

643,750

(£14.29)

£9.2m £28bn

1990 185,000

(£102.16)

615,725

(£30.70)

1,388,250

(£13.61)

£18.9m £45bn

2000 231,351

(£131.83)

672,518

(£45.35)

2,168,916

(£14.06)

£30.5m £52bn

2008 256,622

(£275.50)

741,356

(£95.37)

5,455,655

(£12.96)

£70.7m £80bn

Bassett, D. 2011. Inside Government Forum. Reform of National Exams: Impact on schools and students. 7 July. One Great George Street, Westminster.

Briefing notes, BN43 (2009), IFS

The race to the bottom What is worse?

• for it to be present but not detect it; the ‘total ignorance’ option

• for the public to think that it occurs but it does not (and we cannot demonstrate that it does not); the ‘innocent imprisoned’ option

• to have it, to not be committed to quantifying it, nor to having policy to arrest it; the ‘defiantly irresponsible’ option

A key claim regarding regulation is removal and/or prevention of competition on standards

A new look at standards:

What should we want from GCSEs and A levels?

Presentation by Tim Oates

Group Director

Assessment Research and Development (ARD)

13 October 2011