gavin sim, naveed malik, phil holifield
DESCRIPTION
Strategies for large-scale assessment: an institutional analysis of research and practice in a virtual university. Gavin Sim, Naveed Malik, Phil Holifield. Introduction. Background of VU Infrastructure Strategy Admissions Assignments Examinations Future Strategy Conclusions. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/1.jpg)
1
Strategies for large-scale assessment: an institutional analysis of research and practice in a virtual university
Gavin Sim, Naveed Malik, Phil Holifield
![Page 2: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/2.jpg)
2
Introduction
Background of VU Infrastructure Strategy
– Admissions– Assignments– Examinations
Future Strategy Conclusions
![Page 3: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/3.jpg)
3
Background Virtual University Government Funded Project Courses delivered via satellite television and a
Learning Management System (Intralearn) Utilise lead academics to develop courses Students attend Learning Centres across
Pakistan to view programmes and use the Internet
Evidence that the needs of learners in remote areas can be served via local learning centres (Mayes, 2001)
![Page 4: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/4.jpg)
4
Pak Institute of Management & Computer Science – Lecture
![Page 5: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/5.jpg)
5
Pak Institute of Management & Computer Science - Lab
![Page 6: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/6.jpg)
6
Background - Cont.
Current Cohort 1700 capabilities of 30750 Universities in Pakistan full year course
followed by exams VU this strategy not feasible, students need
to be able monitor progress and obtain feedback
Marking time would be 212 days for typical exam scripts
New Strategy required utilising the technology
![Page 7: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/7.jpg)
7
Infrastructure Guidelines highlight need to identify technical
issues (Stephens, Bull et al. 1998) VU identified, distributed cohort - connectivity,
scalability, security & authenticating users Academic Intranet Courses and assessment delivered via the LMS Intralearn requires a ColdFusion backend and a
DBMS which Microsoft SQL Server is used Intralearn and SQL-Server are hosted on
separate servers creating a 3-Tier approach
![Page 8: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/8.jpg)
8
Infrastructure - Cont.
![Page 9: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/9.jpg)
9
Infrastructure - Cont. Scalability an Issue Max bandwidth between Learning Centres and VU
Hub 2Mbps can increase to 155Mbps The LMS needs an adaptable architecture Custom built software has been developed for
submitting the assignments (http upload). Asynchronous assessment is required due to the
variability in connectivity between centres Students can use any software - this is then
encrypted, renamed and forwarded to the tutor for grading
![Page 10: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/10.jpg)
10
Infrastructure – Cont.
![Page 11: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/11.jpg)
11
Infrastructure - Cont.
Delivery of exams - security issues Security flaws in sending emails such as
authentication and encryption (Hatton, Boyle et al. 2002)
Due to student numbers an asynchronous approach required
Software developed and deployed at centres Exam sent to centres encrypted and placed on
local server Same file returned encrypted after the exam
![Page 12: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/12.jpg)
12
Strategy - Admissions Admissions scheduled for spring and autumn Test administered to 5000 students comprising
MCQ and essay questions. Marking outsourced to National Testing Centre Found no advantage in using this approach due
to bias for those exposed to testing before Test anxiety could adversely effect their grades
(Cassady & Johnson 2002) Admissions based on public examination results
![Page 13: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/13.jpg)
13
Strategy – Assignments
Account for 15% of marks and students usually have between 5-10 per semester.
Regular testing is vital (Croft, Danson et al. 2001)
A series of bad marks can lead to a downward spiral in learning and confidence (Taras, 2002)
The assignments are structured to build student confidence
CS101 first assignment send an email and use the discussion board
![Page 14: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/14.jpg)
14
Strategy – Assignments Cont.
Submission of assignments deadline plus one day
Academics are responsible for designing the assignments
Essay style formats usually used rather than objective testing
Tutors responsible for marking 250-300 assignments per week
Number of tutors is proportional to the number of students enabling scalability
![Page 15: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/15.jpg)
15
Strategy – Assignments Cont.
Solutions are posted on LMS no feedback is provided
Self-reflection against a model answer enables students to gain personal insight of their understanding (Peat, Franklin et al. 2001)
Can ask questions concerning assignments on moderated discussion boards
![Page 16: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/16.jpg)
16
Strategy - Exam
Two exams per module mid-term 35% and final exam 50%
Examiners are responsible for authenticating student ID at learning centres
Exams consist of both MCQ and Essay style questions
MCQ issues surrounding guessing (Harper, 2003)
Essays subjectivity with different markers
![Page 17: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/17.jpg)
17
Strategy – Exam Cont.
Having varied question styles help produce fairer test
MCQ marked automatically whilst essays are marked by tutors
The students type their essays improving legibility for the tutors
This method enables the full range of cognitive skills to be tested as defined by Bloom (1956)
Debate whether objective testing can test higher cognitive skills in all subject domains
![Page 18: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/18.jpg)
18
Strategy - Exams Cont.
In the majority of instances Synthesis and Evaluation promote divergent thinking and answers cannot be determined in advance (Bloom 1971)
Higher Cognitive Skills tested in Linguistics (Reid 2002)
Results are posted on the LMS Any queries are directed to staff within the VU
![Page 19: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/19.jpg)
19
Future Strategy
Group projects planned for final year Students will work with others outside their
geographical area Tutors responsible for grading projects Explicit guidelines will need to be drawn up Ways to overcome gaps in face-to-face
presentations and lab demonstrations are sought
![Page 20: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/20.jpg)
20
Conclusions
Infrastructure and strategy need to be developed in parallel
A strategy of combining CAA and tutor marked assessments is feasible
The scalability of the strategy and infrastructure should enable successful future expansion
Future analysis and research in the field of CAA may produce solutions that overcome some of the drawbacks
![Page 21: Gavin Sim, Naveed Malik, Phil Holifield](https://reader033.vdocuments.site/reader033/viewer/2022050806/56815d70550346895dcb795f/html5/thumbnails/21.jpg)
21
Questions