the effects of naplan a preliminary report for wappa website
DESCRIPTION
This is a copy of my presentation I gave at the Western Australian Primary Principals Conference June 20, 2012.TRANSCRIPT
The Effects of NAPLAN: A preliminary report on teacher perceptions
WAPPA Conference 2012Dr Greg Thompson
Introduction
• In thinking about how to begin speaking to a group of principals, I am always mindful of the words of Brookfield:
The more I know about their work, the more I feel humbled by their abilities … If I know too much about who they are and what they have done as educators, I start to say to myself ‘what on earth can I say to these people that has any chance of being taken seriously, or considered important, by them? (1994, p.207).
• I am particularly conscious of the erroneous assumption often made about expertise
• I am no expert on NAPLAN, rather I see myself as collecting, organising, making the voices of experts heard
Why NAPLAN?
• Began out of my PhD work on how the concept
of a ‘good student’ operates in high schools
• I thought about how Education Policy is ‘done’ to people, how it impacts on those in schools, and how we seem to have lost those mechanisms that ask how policy is working
• I also wondered how policy ‘actors’ (teachers, principals, school communities) engaged with NAPLAN, what strategies they use and why
• No longer any mechanism for shared conversations between schools, universities, bureaucrats, politicians etc
NAPLAN – My Position
• Broader Federal policy platform: accountability/transparency
• Laudable aim: Improving the equity of education outcomes
• Heavily influenced by USA and UK approaches to education.
• Controversial and very difficult to quantify and/or qualify
• Its impacts require scrutiny, coercive approaches to equity may have the opposite effects
An Overview of the Research
• In 2011 the Australian Research Council awarded me a DECRA to look at the effects of NAPLAN on school communities
• $375,000 over three years
• Murdoch co-investing around $220,000
• Only 9% success rate with only 3 of the DECRA’s in Education
• Three phases to the research:
1. A survey of teachers in SA and WA in 20122. Embedded fieldwork in WA schools in 20133. Embedded fieldwork in SA schools in 2014
What makes a standardised test ‘high-stakes’?
• Standardised tests arenot new (WALNA, TAE, TEE ...)appropriate summative measure of large populationsnever ‘perfect’, always design issuesnot necessarily high-stakes
What makes a standardised test ‘high-stakes’ is how the data collected is used
If some (parents) vote with their feet, that's exactly what the system is designed to do
Interesting stories…
Opponents of NAPLAN would deny parents and students information that sits in a bigger picture. They would deny schools that chance to identify others from which they might usefully learn. Worse, they claim that NAPLAN is creating "test-driven schooling". That is nonsense. If teachers burden students with numerous practice tests they are wasting time for students and themselves
NAPLAN will never work as a diagnostic test. A diagnostic test should be administered by the teacher so that they can take action the next day. No national survey can return all of that the next day very easily. There are ways in which we can provide resources—for example, in computer adaptive tests that can provide feedback instantly. Perhaps in the future those sorts of resources can be made available. There is a lot of interest in doing that, but that is not a part of NAPLAN at this stage.
It needs to be made clear to schools and teachers that excessive test practising ahead of NAPLAN is unnecessary. While it helps to be familiar with the structure of the tests, carrying out endless practices should not be encouraged. NAPLAN matters, but it is not the be all and end all.
US and UK: International impacts?
Underperforming schools can be closedStudents can be ‘held back’Teachers have their value-added measures published/ranked
Underperforming schools can be closedFunding tied to league tablesTeachers can have their employment terminated based on test results
(Polesel, Dulfell, Turnbull, 2012)
NAPLAN?
• What makes NAPLAN high-stakes in Australia?publication of the results of the NAPLAN program on the MySchool
website (Lobascher, 2011, pp. 9-10) associated media coveragerealities of enrolments, encouraging parents “to vote with their feet”,implementation of the recommendations relating to Queensland’s 2008
poor NAPLAN performance (Lingard, 2010)
Some more dire consequences in USA/UK.
May be interesting to see what happens in 10 years here.…
Is there another side to the research?
Research is rarely one-sided.
A number of studies have found high-stakes
testing has an educational benefit. They suggest it improves:learning, because what is expected is clearly understoodefficiency because teachers/schools work harderincreased comparability and cohesion across school systems and contexts.
Published (generally) in management/ administration/economic journals
Survey Tool
• The survey consists of 5 ‘sections’ that link to explore teacher perceptions about the impact of NAPLAN. These are:DemographicsStressSelf-EfficacyImpact on their curriculum and pedagogyAchievement Goal Orientation
• It is open to all teachers in WA and will close at the end of June… (www.effectsofnaplan.edu.au)
• The more participants the more balanced/representative the information
Teacher Perceptions
“You are asking teachers to make value judgements!”Absolutely. Teachers are actually quite good at making professional judgements.
“Steps forward?”Valuing professional judgements of educators is a hallmark of Finnish approached to schooling
But need to triangulate ...
Impacts on Curriculum and Pedagogy
• 14 questions in this section
• Derived from Lingard et al (1990) idea of ‘productive pedagogies’
• 20 observable characteristics of classroom pedagogy that represented ‘best practice’
• Many of these ideas we are now familiar with, but in the early 1990s were considered ‘revolutionary’
But…• Two main areas of criticism of the survey method of research:
Questions lead the responsesOnly certain people respond, generally those that already have
strongly held opinions at a certain point in time
• Possible criticism: teachers responding are probably those with the most extreme/negative views about NAPLAN
• If respondents are representative of certain pre-existing attitudes, interesting patterns emerge
• The sub-set of questions examined in the following pages refer to impact on curriculum and were identified as statistically related
To summarise…
1. Across WA and SA teachers in low-SES schools perceive that NAPLAN has had a greater impact on curriculum in their classroom
2. Teachers in Government schools perceive greater impacts than those in Catholic schools and significantly more than Independent schools
3. Teachers in schools in WA perceive a greater impact on curriculum than their SA counterparts
The next challenge is to explain why.
Given the research addressed, it may be that NAPLAN is having an opposite effect than intended – widening the achievement/equity gap.
What other information will the survey provide?
• Next 6-12 months, research that examines:Teacher stress across demographicsA comparison of positive and negative impacts of NAPLAN across demographicsImpact on pedagogy across demographicsImpact on teacher self-efficacy across demographics
• Applied to ACARA Board to access all of the NAPLAN data for all schools in WA and SA from 2009-2011. This will enable a comparison point to the teacher perceptions.
• Beginning Phase 2 of the research: case studies undertaken in WA school communities
Where to next?• Three future possibilities for NAPLAN (as I see it ...):
Nothing changes, or we get more testing (canvassed at the Federal level, history from US/UK)
Testing side of NAPLAN is kept, but the elements that make it so ‘high-stakes’ are removed or have their impact lessened
NAPLAN is removed
• “In what ways will schools respond to NAPLAN beyond that of excessive preparation and a return to teacher-centred pedagogies?”
• Interested in comparing teacher perceptions with parents, principals and students.
And finally…
• I am no expert on NAPLAN, I collect the stories of those people who understand it the best.
• With that in mind, what could you suggest as interesting possibilities to look at in schools in WA?