improving assessment, improving learning ken greer, executive director (education), fife council cem...
TRANSCRIPT
Improving Assessment, Improving Learning
Ken Greer, Executive Director (Education), Fife Council
CEM Conference, Glasgow
24th March 2011
Implementing Building the Curriculum 5
How the world’s best-performingschool systems come out on top
“All of the top-performing and rapidly improving systems have curriculum standards which set clear and high expectations for what students should achieve.
“High performance requires every child to succeed.
“The only way to improve outcomes is to improve instruction.
“All of the top-performing systems also recognise that they can not improve what they do not measure.”
The McKinsey Report, September 2007
Assessment:It’s about asking questions
1. What’s assessment for?
2. What system are we working with?
3. How are we doing it?
4. How do we make sure we are all talking about the same standard?
5. Which unintended consequences do we want to avoid?
6. How do we put all this together and make it work in Fife (the unashamedly Chauvinistic).
1. What’s assessment for?
– to support learning;
– to give assurance to parents and others about learners’ progress; and
– to provide a summary of what learners have achieved, including through qualifications and awards, and to inform future improvements.
-Building the Curriculum 5
1. What’s assessment for? (2)The Assessment Reform Group (ARG)
The use of assessment : to help build pupils’ understanding, within day-to-day
lessons to provide information on pupils’ achievements to those
on the outside of the pupil-teacher relationship: to parents (on the basis of in-class judgements by teachers and test and examination results) and to further and higher education institutions and employers (through test and examination results)
data to hold individuals and institutions to account, including through the publication of results which encourage outsiders to make judgments on the quality of those being held to account.
2. What system are we working with?
Building the Curriculum 3 (June 2008) Curriculum for Excellence: Es and Os CfE BtC 5 A framework for assessment: recognising
achievement, profiling and reporting (December 2010) CfE BtC 5 A framework for assessment:
understanding applying and sharing standards in assessment for CfE: quality assurance and moderation (October 2010)
The NAR http://www.ltscotland.org.uk/nationalassessmentresource/
51,000+ teachers (4000 in Fife)
Arrangements for
•Assessment
•Qualifications
•Self-evaluation and accountability
•Professional development
to support the purposes of learning
3. How are we doing it?
CfE aspirations delivered (SLCIRCEC) CPD; support; challenge; reporting Knowing the limitations of various approaches to assessment Measuring what we value Working together to moderate/define standards led by expert
practitioners Monitoring progress, monitoring value-added Motivating: defining the bar Analysing; benchmarking; supporting Giving account and holding to account Finding a manageable way: economy, efficiency, effectiveness Milestones, not millstones
4. How do we make sure we are alltalking about the same standard?
Trust/professionalism The primacy of individual teachers’ judgements is at the
heart of the assessment system in Scotland, supported by moderation at local authority level and across authorities
A National Assessment Resource (NAR) to support teachers as they come to judgements about learners’ progress
Outcomes and experiences, but not performance criteria Prior performance/other information? The car with no speedometer, no odometer and no petrol
gauge
A view from another country
“Progress also relies on the need to retain clear accountability through testing. This means at the end of primary school just as much as at the end of secondary.”
Gordon Brown, quoted in TES, 30/10/2009
“In the less successful secondary schools, the limited use of assessment data on pupils on transfer to Year 7 led to insufficiently challenging targets for some pupils.
“In raising the attainment of learners in literacy who are most at risk of not gaining the skills they need for successful lives, the factors identified from visits on this survey included sharp assessment of progress in order to determine the most appropriate programme or support.”
Removing Barriers to Literacy, Ofsted 2011
5. Which unintended consequencesdo we want to avoid?
What we want to avoid: assessment which does not support learning,
directly or indirectly de-motivation of any learner self-fulfilling prophecies ‘high stakes’ testing league tables false comparisonsWhat we want to promote: Improvements in learning, teaching and
performance (SLCIRCEC)
6. What’s are we doing in Fife?Fife’s performance culture
Appropriateinformation
Strategy for improvement
Collegiate approach
Better understanding
The Fife way
Importance of the……….
• right strategy
• right culture
• right information
• right interpretation
• right results i.e. positive impact on performance
Themes
Concentration on Impact: relentless focus on outcomes
Culture: Need to develop a strong performance culture
Context: Need to understand underlying performance issues in an appropriate context
Clarity: Need to use appropriate information to identify performance issues
Context What does national data tell us?
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7 8 9 10
SIMD decile
Le
av
ers
en
teri
ng
HE
(%
)
0
2
4
6
8
10
12
14
1 2 3 4 5 6 7 8 9 10
SIMD decileL
ea
ve
rs e
nte
rin
g t
rain
ing
(%
)
More deprived More affluent
Challenging a deterministic view
“…the PISA scores of the top-performing countries show a low correlation between outcomes and the home background of the individual student.”
The McKinsey Report, September 2007
However, in Fife (Scotland?), social context has a strong relationship with attainment and other educational outcomes, including destinations
Educational outcomes vary with social context across the social spectrum
We need to understand this in order to address it.
Raising attainment … for all
Scotland: leavers entering HE (%)
0
10
20
30
40
50
60
70
Le
av
ers
en
teri
ng
HE
(%
)
Fife: attainment by end of S4
100
150
200
250
Mos
t dep
rived
Decile
2
Decile
3
Decile
4
Decile
5
Decile
6
Decile
7
Decile
8
Decile
9
Leas
t dep
rived
Av
era
ge
SQ
A t
ari
ff p
oin
ts
BenchmarkingExample: comparator authorities
Authority 1 is a comparator authority for Authority 2 – rated as “very close” by HMIE
In 2010, 9.4% of Authority 1 secondary pupils were FME, compared to 9.7% of Authority 2’s secondary pupils
2.2% of children in Authority 1 live in the SIMD 15% most deprived areas in Scotland, compared with 3.3% in Authority 2
Example: comparator authorities
In 2010, 38% of Authority 1 pupils achieved 5+ at level 5 by the end of S4, as compared with 65% in Authority 2
What accounts for this difference?
Raising attainment … for all
Performance management needs to account for the impact of social context and other relevant factors
But … current school performance measures focus attention on the most deprived (e.g. FME, SIMD 15%)
Need the best data to fit the issuenot the best fit to the available data
Some of the best data …
CEM assessments from the University of Durham (PIPS, INCAS, MidYIS, SOSCA)
Standardised to a national level of performance and comparable across stages
Provides a coherent view of performance at local authority, establishment, curriculum area & class levels
CEM assessment data
PIPSP1
PIPSP3
PIPSP5
PIPSP7
SOSCAS2
SQAS4, S5, S6
Tracking
Individual level
Performance information
School, curriculum area and class level
Looking across a cohort: Fife
R2 = 0.99
100
150
200
250
Ave
rag
e S
QA
S4
tari
ff p
oin
ts
Most deprived deciles Least deprived deciles
The same cohort at P7: Fife
R2 = 0.98
40
45
50
55
60
Ave
rag
e P
IPS
P7
sco
re
Most deprived deciles Least deprived deciles
Continuous improvement … “Value added” measures
At Fife level there is a strong correlation between performance in PIPS (at stage P7) and SQA (by S4) when viewed by social context.
This is related to outcomes …
More affluent
More deprived
R2 = 0.98
100
150
200
250
40 50 60
PIPS P7 average
SQ
A S
4 av
erag
e
Continuous improvement … “Value added” measures
The red lines separate levels of attainment most common amongst those who enter …
HE
FE, Employment
Unemployment
More affluent
More deprived
R2 = 0.98
100
150
200
250
40 50 60
PIPS P7 average
SQ
A S
4 av
erag
e
Conclusion: outcomes and social context
There is substantial evidence that educational outcomes vary across the social spectrum
Current approaches to measuring school performance do not adequately reflect this relationship
Conclusion: local data sources
Local sources of information (e.g. CEM data) can give valuable additional insight
This can help to understand year-on-year variations in performance
This can provide “added value” measures:– Across the social spectrum
– Within a given cohort
– By subject area
Conclusion: national data sources
There is a lack of relevant data across the social spectrum
E.g. there is no national data available on the SIMD profile of each education authority (EA) or school
Relevant national data are vital for real understanding and improvement of:– EA and school performance
– outcomes for young people