sqa adapted digital question papers: the story so far…. paul d. nisbet call scotland university of...
TRANSCRIPT
SQA Adapted Digital Question Papers:
The story so far….
Paul D. NisbetPaul D. Nisbet
CALL Scotland CALL Scotland
University of EdinburghUniversity of Edinburgh
Adapted Digital Papers2006 (pilot)
2007 (pilot) 2008 2009 2010
Number of centres 8 12 46 73
Number of candidates 34 80 204 422
Number of requests 65 265 514 1,167 1,962
Number of candidates / centre 4.25 6.67 4.27 5.78
Number of requests / candidates 1.91 2.50 2.51 2.76
Requests for Adapted Digital Papers 2006-2010
2009 requests by local authorityAuthority Requests Authority Requests
Fife CouncilFife Council 149149 The Moray CouncilThe Moray Council 3131
The City of EdinburghThe City of Edinburgh 122122 Highland CouncilHighland Council 2222
FalkirkFalkirk 9797 Shetland Islands CouncilShetland Islands Council 1818
Dumfries and GallowayDumfries and Galloway 8989 North Ayrshire CouncilNorth Ayrshire Council 1616
AberdeenshireAberdeenshire 6565 North Lanarkshire CouncilNorth Lanarkshire Council 1515
Dundee CityDundee City 6262 Renfrewshire CouncilRenfrewshire Council 1313
South AyrshireSouth Ayrshire 6060 East Dunbartonshire CouncilEast Dunbartonshire Council 77
Perth & KinrossPerth & Kinross 5959 East Renfrewshire CouncilEast Renfrewshire Council 44
Aberdeen CityAberdeen City 5252 Midlothian CouncilMidlothian Council 44
West LothianWest Lothian 4545 Western Isles CouncilWestern Isles Council 22
City of GlasgowCity of Glasgow 4242 East Ayrshire CouncilEast Ayrshire Council 11
Scottish BordersScottish Borders 4141 South Lanarkshire CouncilSouth Lanarkshire Council 11
Argyll and ButeArgyll and Bute 3838
StirlingStirling 3636 FE CollegesFE Colleges 2121
InverclydeInverclyde 3333 Independent Secondary Independent Secondary SchoolsSchools
2222
2009 requests by schools
2009 Centres Requests Local Authority
Queen Anne High School 122 Fife
The Royal High School 82 City of Edinburgh
Bo'ness Academy 68
Stranraer Academy 62 Dumfries & Galloway
Prestwick Academy 60 South Ayrshire
Kinross High School 59 Perth & Kinross
Mintlaw Academy 48 Aberdeenshire
Ashcraig School 37 Glasgow City
Campbeltown Grammar School 37 Argyll & Bute
Peebles High School 35 Scottish Borders
Aboyne Academy 34 Aberdeenshire
St Columba's High School (Gourock) 33 Inverclyde
St Paul's RC Academy 32 Dundee City
Holy Rood High School 30 City of Edinburgh
Buckie High School 29 Moray
2009 requests by subject (subjects with requests > 10)
SubjectNumber of requests Subject
Number of requests
EnglishEnglish 284284 FrenchFrench 3939
BiologyBiology 9898 Modern StudiesModern Studies 3737
HistoryHistory 9292 Physical EducationPhysical Education 3636
Computing StudiesComputing Studies 9191 MathematicsMathematics 2929
GeographyGeography 7676 Business ManagementBusiness Management 2828
ChemistryChemistry 5050 DramaDrama 2828
Craft & DesignCraft & Design 5050 Home EconomicsHome Economics 2222
AdministrationAdministration 4848Mathematics: Maths 1, 2 Mathematics: Maths 1, 2 and 3and 3 1212
PhysicsPhysics 4242 GermanGerman 1111
Reasons for requesting ADPs, 2009
Candidates' Difficulties
Physical or Motor Impairment 143
Other Moderate Learning Difficulty
65
Autistic Spectrum Condition 39
Visual Impairment 30
Dyslexia 474
Other Specific Learning Difficulty
244
Physical Health Problems 61
Social Emotional Behavioural
Difficulty 37
Language or Speech Disorder
32
Learning Disability 38
Pupils’ comments 2006 & 2007
Prefer it to using a
scribe. Easier to give
answers.
Easy to do. You can see both the questions and the text at the same time.Enjoyed using
it!
I would prefer to have split screens so that I could see questions and tables together instead of having to refer to another page.
Pupils’ comments 2006 & 2007
Papers are simpler to use. You do not have to fiddle about with different papers. By switching windows, you can see both the question and the text at the same time.
Because it is much easier to use than a reader
Staff evaluations 2006 trials
0
1
2
3
4
5
6
Confidence Independence Motivation Expertise
Traditional papers Digital Papers
Pupils’ results with digital papers
““Candidates’ results from digital papers are Candidates’ results from digital papers are similar to their teachers’ estimates”similar to their teachers’ estimates”
““there appears to be little difference between there appears to be little difference between [marks awarded for] entries using digital papers [marks awarded for] entries using digital papers and the other entries sat by the same and the other entries sat by the same candidates”candidates”
Nisbet, P.D., Shearer, N., Balfour, F., Aitken, S., (2006) Nisbet, P.D., Shearer, N., Balfour, F., Aitken, S., (2006) SQA Adapted SQA Adapted Examination Papers in Digital Format: Feasibility Report 2005-2006Examination Papers in Digital Format: Feasibility Report 2005-2006. .
www.adapteddigitalexams.org.ukwww.adapteddigitalexams.org.uk
Reliability ““Digital exams were the only part of the 2007 Digital exams were the only part of the 2007
diet which did not cause significant problems – diet which did not cause significant problems – they simply worked. Probably this was assisted they simply worked. Probably this was assisted by more stringent attention to detail, adequate by more stringent attention to detail, adequate risk assessment, etc.”risk assessment, etc.”
““I personally would like to see digital exams as I personally would like to see digital exams as the default choice for pupils with physical the default choice for pupils with physical disabilities, and paper, helpers, scribes etc as disabilities, and paper, helpers, scribes etc as options which would need to be specially options which would need to be specially requested, because I believe they are requested, because I believe they are empowering, less difficult to administer and cost-empowering, less difficult to administer and cost-effective.”effective.”
Accommodation““We had 2 rooms set up with We had 2 rooms set up with double sized tables to allow a double sized tables to allow a comfortable amount of room comfortable amount of room for laptop and papers. Each for laptop and papers. Each candidate was allocated a candidate was allocated a table which was the same for table which was the same for each exam. All tables were each exam. All tables were numbered and we made use of numbered and we made use of screen dividers which created screen dividers which created a booth effect and cut down a booth effect and cut down distractions and gave a certain distractions and gave a certain amount of privacy in the small amount of privacy in the small classroom. We had 9 classroom. We had 9 candidates sitting the exam in candidates sitting the exam in digital format and 2 writing in digital format and 2 writing in the same room for the the same room for the Standard Grade English exam. Standard Grade English exam. Candidates names and Candidates names and numbers were on the desks, numbers were on the desks, and times.”and times.”
% of AA candidates using ADPs
(12 schools 2006-08)
Ashcraig
Broughton High
Buckie H
igh
Campeltown H
igh
Graeme High
Holy Rood H
igh
Inverness H
igh
Kinross High
Prestwick
Academy
The Roya
l High
Stranraer A
cademy
Uddingston
2006
2008
0%
10%
20%
30%
40%
50%
60%
70%
2006
2007
2008
2008 requests to support writing (46 centres using ADPs)
Use of ICT
Digital Papers
Scribe
Candidate Signs Responses
Use of tape recorder for responses
Transcription without correctionTranscription with
correction
Calculator
PA Referral
“We really appreciate this format of exam paper. The pupils are generally much happier to be independent rather than depend on scribes and readers. This year all our S3 and S4s sat the SG English examination at the same time. If we had had to provide readers and/or scribes for this we would not have been able to staff it.”
Support for writing 2007-08(12 centres using ADPs)
ADPICT
Scribe
2007
20080%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Support for reading(data from 18 centres in 2008)
Reading support 2008
Reader, 894
Question Paper signed to
candidate, 1
Coloured Paper, 74
Text reading software, 71
Enlarged Print, 63 Braille paper, 2
Text-reading software2007-2008
Ashcraig School
Prestwick Academy
Buckie High School
Campbeltown Grammar School
2007
20080%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Use of text reading software 2007 - 2008
2007
2008
Use of readers 2007-2008
Ashcraig School
Prestwick Academy
Buckie High School
Campbeltown Grammar School
2007
20080%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Use of readers 2007-2008
2007
2008
Cost analysis:School ‘H’ English Standard Grade, 5 pupils using ADPs
instead of reader/scribes
RoomsRooms TimeTime Reader/scribe Reader/scribe timetime
Reading Reading GeneralGeneral
55 1 hour1 hour 5 hours5 hours
Reading Reading CreditCredit
55 1 hour1 hour 5 hours5 hours
Writing F/G/CWriting F/G/C 55 1.5 hours1.5 hours 7.5 hours7.5 hours
TotalTotal 1515 3.5 hours/ pupil3.5 hours/ pupil 17.5 hours17.5 hours
School ‘H’ Standard Grade: digital papers for 5 pupils
5 pupils, 5 computers, 1 room5 pupils, 5 computers, 1 roomPreparation time (SfL): 1 hour?Preparation time (SfL): 1 hour?Technician time: Technician time: 2 hours?2 hours?Running time (SfL): Running time (SfL): 3.15 hours?3.15 hours?Total:Total: 7.15 hours7.15 hours
School ‘H’ readers/scribes vs. digital
Readers / Readers / scribesscribes
Digital Digital PapersPapers
SavingSaving
Staff hoursStaff hours 17.517.5 7.157.15 10 hours10 hours
Staff cost @ £25/half dayStaff cost @ £25/half day £250£250 £143 *£143 * £107£107
Staff cost @ £20/hourStaff cost @ £20/hour £350£350 £143£143 £207£207
Staff cost @ £200/dayStaff cost @ £200/day £1000£1000 £143£143 £857£857
RoomsRooms 55 11 4 rooms4 rooms
School H buys in all readers/scribes at £25 per half day.
The school saved £1000 by using ADPs in 2008.
* Cost calculated at £20/hour
Costs of readers/scribes
16,554 reader requests; 14,197 scribe 16,554 reader requests; 14,197 scribe requestsrequests
~ 30,000 individual exams~ 30,000 individual exams~ 60,000 hours~ 60,000 hoursSay average £20/hour for reader/scribe?Say average £20/hour for reader/scribe?
= £1,200,000= £1,200,000Say £10/hour for invigilator?Say £10/hour for invigilator?
= £600,000= £600,000TOTAL = TOTAL = £1.8 m£1.8 m in Scotland per annum in Scotland per annum