school of education investigating influences on post-16 progression in science in england using...
TRANSCRIPT
School of Education
Investigating influences on post-16 progression in science in England using quantitative national data
Matt Homer
UK Science Education Research Conference –
Researchers sharing with researchers 2-4 July 2012
National Science Learning Centre,
University of York
School of Education
Some background issues…
• In the UK we have a lot of national assessment data.
• How can and should it be used?– Researchers, policy makers, schools, others?
• What are the limitations of findings from such data?
• The context: what does post-16 science participation look like – what can national data tell us about key influences on this?
– How might curriculum reform change patterns of participation?
2
School of Education
Overview of talk
• Policy and project (EISER) background• National data sources – the NPD• Known influences on post-16 science participation• Appropriate statistical methods• Results: Descriptive analysis – gender/SES/14-16
pathway• Results: Modelling participation• Conclusion/Discussion
– implications for policy/further research?
33
School of Education
Policy reformA new science curriculum for 14-16 year olds introduced in England in 2006
• Flexibility: a greater variety of ‘routes’ through 14-16 (KS4) science.
• A focus on teaching about the nature of science and socio-scientific issues – How science works
• Enhanced presence of vocational science courses (‘applied sciences’).
Also,
• ‘Entitlement’ to Triple award (separate sciences at KS4) from 2008 – big increases in students
4
School of Education
http://www.education.leeds.ac.uk/research/projects/enactment-and-impact-of-science-education-reform-eiser
Our project: Enactment and Impact of Science Education Reform (EISER)Mixed methods, 2008-2011: jointly funded by the Gatsby Charitable Foundation and the Economic and Social Research Council
This study examines school responses to this major curriculum reform. A particular focus is teacher enactment of the science curriculum in the classroom. The study is also investigating the initial impact of these reforms on student achievement, attitudes towards science education and participation in post-compulsory science courses.
Document analysis, interviews with teachers and students, National data analysis
Jim Ryder, Indira Banner, Matt Homer, Jim Donnelly…
5
School of Education
Immediate Longer Term
Increase student interest in their science education
Improve student attainment as measured through external examinations
Support students in engaging effectively with science-related issues as citizens
Increase post-compulsory participation in science education
Ensure adequate supply of scientists/engineers
Increase the employability of students
Improve social mobility and inclusion
Multiple aims of reform
Ryder and Banner (2011)
6
School of Education
Data source: the National pupil database (NPD)What is in it?
•All pupils in state-funded schools in England.
•Pupil level assessment data (primary to post-16)
•Pupil level personal data (e.g. gender, ethnicity, SES...).
•School level data (phase, selection policy, governance,…)
•Files linked by a unique pupil ID number.
•Large files: e.g. 600, 000 per cohort so KS4 results file – 6 million records for a particular year – 1 per qualification (GCSE…), i.e. multiple records per pupil.
•A powerful resource – the national picture and how it is changing over time.
7
School of Education
Use of NPD datasets in EISER• What are the patterns of participation and
attainment across KS4 and KS5 science courses?• How is this changing over time? • What are the influences on participation and
attainment?
Five successive KS4 cohorts• Two pre-2006 reform: 04-06, 05-07• Three post-reform: 06-08, 07-09,08-10
• Initial focus on KS4 – longitudinal change
Have recently started focussing on post-16…
8
School of Education
Our post-16 analysis is limited for two main reasons:
•NPD AS data is problematic (~20% missing due to cashing/not cashing in issues) – fixed(?) in newest data
•As of Jan 2011, we had only one post-reform full A-level cohort of data – hence it is probably too early to see longitudinal ‘impacts’ of the reforms on post-16 participation and attainment.
•Have investigated first post-reform cohort...(KS4: 06-08, A-Level 2010)
•Work in progress…
Problems with post-16 data
9
School of Education
Post-16 science – influences on participation
These are some known factors in uptake:•attainment at 16•14-16 pathway – Triple award vs. Dual award vs ‘Other’•Gender – e.g. biology vs. physics•Socio-economic status – lower groups less likely...
•(Plus teachers, student identity, school policy…)
But…•What are the relative ‘sizes’ of these effects?•How do they vary across the sciences (biology, chemistry, physics) and against other subjects?•To what extent are supposed ‘science’ problems with participation specific to science?
10
School of Education
Methods: progression to eight courses compared
Name of qualification Type of qualification More details of qualification types
Science courses
A-levels (formally Advanced Level General Certificate of Education) are usually studied over a two-year period, and entrance to university is normally predicated on sufficiently high achievement in these qualifications.
Applied A-levels are A-levels with a vocational emphasis.
BTEC National Awards are equivalent to A-levels in terms of academic value, but have a vocational focus.
Biology A-level
Chemistry A-level
Physics A-level
Single Award Applied Science
Applied A-level –vocationally orientated
BTEC National Award Applied Science
A vocationally orientated qualification
Comparator courses
Mathematics A-level
Psychology A-level
History A-level
11
School of Education
Methods – two approaches1. Descriptive statistics: comparing participation rates by
• Gender, SES and 14-16 pathway separately• Also, controlling for prior attainment at 16 in science
2.Multi-level modelling: predicting participation for each course based on gender, SES and 14-16 pathway plus:
• attainment in science and maths at 16, • socio-economic status (2 measures - free school meal, FSM;
and Income Depravation affecting children index, IDACI)• whether 14-16 school also teaches to 18
This modelling gives measures of the INDEPENDENT effect of each variable, and also gives an indication of how important schools are in determining participation.
12
School of Education
Descriptive results: overall participation post-16
13
Number and percentage of full cohort at 16 taking each qualification
Main messages
• Participation rates are generally quite low (a few per cent)
• The vocational routes are small
(A-levels etc in 2010)
School of Education
Red: all students doing the course
Blue: all students doing the course who achieved a (mean) grade ‘B’ in science at 16
Main messages
• Wide variation in participation rates across both sciences and non-sciences – e.g. biology/physics, maths/psychology
• The role of KS4 attainment – it certainly does not account for the differences
– within courses red bars similar to blue in length
Descriptive results: participation by genderPercentage of female students within courses
14
School of Education
Main messages
• Red: wide variation in participation rates across both sciences and non-sciences – but generally low – vocational courses the exception
• Blue: Controlling for prior attainment partly ameliorates the under-representation. But not entirely.
So FSM students are generally less likely to participate (e.g. physics) but this is not a ‘science’ problem per se (e.g. history).
Descriptive results: participation by FSMPercentage of FSM-eligible students within courses
15
School of Education
Main messages
• Red: As we would expect, TA students are heavily represented in ‘traditional’, but not in vocational, sciences (esp. physics)
• Blue: Controlling for prior attainment makes little difference.
Descriptive results: participation by pathwayPercentage of TA students within courses
16
School of Education
Multi-level modelling results
• Coefficients are odds ratio of participating compared to not
• An odds ratio of 1 means no effect on participation , >1 implies more likely to participate…
• All significant at 5% level except for shaded cells
Post-16 course
Odds ratios estimates for participation
Percentage of additional
variation at school level
Mean prior attainment in science at 16
Prior attainment in maths at 16
Gender: Male
Measures of socio-economic
status
14-16 science pathway 14-16
school teaches up to age 18FSM IDACI TA Other
Biology 1.237 1.008 0.652 1.052 1.281 2.550 0.132 1.077 6.6Chemistry 1.260 1.068 1.177 1.313 2.702 3.435 0.138 0.963 11.3Physics 1.188 1.133 7.683 0.971 1.076 2.275 0.168 1.239 9.9Single Award Applied Science
1.074 1.007 0.849 0.846 0.824 0.298 2.106 2.020 69.3
BTEC National Award Applied Science
1.068 0.998 0.838 1.107 2.044 0.708 1.094 1.381 74.1
Mathematics 1.061 1.402 2.368 1.049 1.600 1.370 0.660 1.075 10.2Psychology 1.076 1.034 0.292 0.856 0.780 0.715 0.453 0.967 8.4History 1.115 1.010 0.924 0.653 0.391 0.778 0.501 1.680 9.2
Odds ratio estimates are significantly different from 1 at 5% level except for those shaded. 17
School of Education
Implications for policy?
Odds ratio estimates are significantly different from 1 at 5% level except for those shaded.
• Gender: Girls are ‘missing’ from physics but so too are boys (a little) from biology – does it matter?
• TA tends to encourage participation in science but is this weakening over time? More to do longitudinally
• Students from lower SES backgrounds are not discriminated against in terms of progression to the sciences (once prior attainment is accounted for).– Other subjects are worse e.g. psychology, history
• Vocational sciences are of a different character – and schools play a much bigger role in promoting progression
More to do…what influences attainment?
18
School of Education
Finally…•The NPD is undoubtedly very useful in terms of assessing the macro picture following reform, and tracking longitudinally.
•Also need a mixed methods approach to find out what (and the why) is going on in schools, attitudes science etc.
•More changes working their way through the system
– EBacc, National curriculum review(?), changes to accountability measures (league tables, what counts and how much), O-levels (!)
19
School of Education
References• Banner, I; Donnelly, J; Homer, M; Ryder, J (2010) The impact of recent reforms in
the key stage 4 science curriculum In: School Science Review 92 (339) pp. 101 – 109
• Homer, M., Ryder, Jim & Donnelly, J., 2011. Sources of differential participation rates in school science: the impact of curriculum reform. British Educational Research Journal, pp.1-18.
• Homer, M., Ryder, Jim & Donnelly, J., 2011. The use of national data sets to baseline science education reform: exploring value-added approaches. International Journal of Research & Method in Education, 34, pp.309-325.
• Ryder, J; Banner, I. (2011) Multiple aims in the development of a major reform of the national curriculum for science in England In: International Journal of Science Education 33 (5) pp. 709 – 725
Thank you – questions?
Matt Homer: [email protected] http://www.education.leeds.ac.uk/people/staff/academic/homer
20