Evidence and policy in education

Tom SchullerUniversity of Brno

May 2011

AimsProvide overview of OECD experience

on evidence/policy interface

Illustrate from UK Inquiry into the Future of Lifelong Learning

Offer a menu for reflection

“There is nothing a politician likes so little as to be well informed; it makes decision-making so complex and difficult.” John Maynard Keynes,

The context: a widening lens

• General questioning of outcome measures at different levels, eg on national economic performance

• Issue of trust in official statistics, and in the presentation of public policies

• Massive increase in public access to ‘information’

“ Measuring the size of these wider benefits of learning is an important research priority, where progress requires better measures of people’s characteristics in a range of domains and surveys that follow the same individual over time.”

Stiglitz, Sen and Fitoussi 2009Measuring Economic Performance and Social

Progress, p47

Difference between trust in official statistics and trust in national governments

-30%-20%-10%

0%10%20%30%40%50%

BE CZ

DK D EE EL

ES FR IE IT CY LV LT LU HU

MT

NL

AT

PL

PT SI

SK FI SE

UK B HR R TR

BE – Belgium; BG – Bulgaria; CZ - Czech Republic; DK – Denmark; DE – Germany; EE – Estonia; EL – Greece; ES - Spain; FR – France; IE – Ireland; IT – Italy; CY - Cyprus; LT – Lithuania; LV – Latvia; LU – Luxembourg; HU – Hungary; MT – Malta; NL - The Netherlands; AT – Austria; PL – Poland; PT – Portugal; RO – Romania; SI – Slovenia; SK – Slovakia; FI – Finland; SE – Sweden; UK - The United Kingdom; HR – Croatia; TR – Turkey.

Lessons from OECD educational R&D reviews

General conclusions :• Low levels of investment• Low capacity• Weak research/policy/practice links

Government expenditure on ERD as a percentage of total (public and private, all levels) expenditure on education

Source : OECD, 2009. Data on ERD are for 2007. Data on expenditure on education are for 2006.

Government expenditure on ERD as a percentage of the public expenditure on R&D

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

2007% 2008%

Source : OECD, 2009.

Knowledge networks

Strengths/weaknesses of each link?

Policy-makers Researchers

Practitioners

MediaLeaders

School

boards

Parents Teacher Orgs

Lessons from OECD educational R&D reviews

General conclusions :• Low levels of investment• Low capacity• Weak research/policy/practice linksRecommendations:• Balancing the research portfolio• Accumulation: building a knowledge base • Dissemination and brokerage• Capacity-building

Methodologies and capacities

Methodological debate:Scientific ideal(s) vs. best available Warfare, mutual invisibility or complementarity

Capacity building:– deepening vs. broadening– producers and consumers

Questions:What forms of capacity are most in need of strengthening?How and by whom should this be done?

Brokerage agenciesIssues/functions:• Dissemination: publications, internet,

presentations• Promoting interactivity• Legitimating rigour/quality• Developing cooperation/trust

Questions:Rationales: what are the different functions and of brokerage agencies?Effectiveness: what are their achievements to date?

http://www.tlrp.org/pa

Aims of the Inquiry

The overall goal : an authoritative and coherent strategic framework for lifelong learning in the UK:• Articulating a broad rationale for public and private

investment in lifelong learning;• A re-appraisal of the social and cultural value

attached to it by policy-makers and the public;• Developing new perspectives on policy and practice.

IFLL Final ReportIFLL Strategic Framework for Lifelong Learning

Interim Papers

Thematic Stocktake Sectoral Public Value

Prosperity Employment & Work

Demography and Social Structure

Wellbeing and Happiness Migration and

Communities Technological Change Poverty Reduction Citizenship and Belonging Crime and Social

Exclusion Sustainable Development

Public Sector Investment Private Sector Investment Third Sector Investment Individual Commitment

Participation, over 10 years

Early childhood Schools Further Education Higher Education Local Authorities Voluntary Sector Family learning Cultural institutions Private Training Providers

Poverty Health Crime Wellbeing Equality

Horizon Scanning / Scenario Planning

Outline of Inquiry Papers

Expenditure on costs of learning provision (£ billion), 2007-08

18-24 25-49 50-74 75+0

10

20

30

40

50

60

70

80

90

100

% of current population

% of current expenditure

Projected % of population by 2020

Proposed % of expenditure by 2020

Proposed re-balancing of expenditure by 4 life stages

Base lifelong learning policy on a new model of the educational life course, with four key stages (up to 25, 25-50, 50-75, 75+) Rebalance resources fairly and sensibly across the different life stages Build a set of learning entitlements Engineer flexibility: a system of credit and encouraging part-timers Improve the quality of work Construct a curriculum framework for citizens’ capabilities Broaden and strengthen the capacity of the lifelong learning workforce Revive local responsibility…. …within national frameworks Make the system intelligent

Ten Recommendations

Recommendation 10Make the system intelligent

• State of Learning: 3-yearly stocktake• Benchmark with other countries• Benefit/cost analyses• Experimentation

HERDIS/Tom Schuller 22

A ‘systemic’ approach to knowledge management

A focus on how the stakeholders:- Generate good quality knowledge and

information- Share that knowledge and information

effectively- Work together to improve the knowledge

base and its utilisation

HERDIS/Tom Schuller 23

Engineering a stronger knowledge base(from review of educational research in Hungary)

• Raising the aspirations of educational researchers, with clearer incentives for performance

• A clearly focused drive to take forward the training of doctoral students

• Reform of initial and in-service teacher training to make the profession more capable of absorbing research.

• Strengthening leadership within the sector. • Reform HE management structures to promote a more

strategic approach to knowledge management in the sector.• More emphasis on experimentation and evaluation

The nature of ‘evidence’

• Precision vs ‘certainty’

• Measurable now vs what should count

• Timeframes

• Political demand: narrative and decision

Future challenges

• Defining an ‘evaluation culture’• Selecting key capacities• Building social capital into research strategies• Complementarity/accumulation: promoting

mixed methods to strengthen the knowledge base

References

Evidence in Education: Linking Research and Policy (CERI/OECD 2007)

New Challenges for Educational Research (CERI/OECD 2003)

www.lifelonglearninginquiry.org.uk

tomschuller@longviewuk.com