online assessment

Online Assessment 2013 - Theory into practice Seminar

Online Assessment Theory into Practice seminar 7 March 2013

1

James Lamb @james858499

[email protected]

Seminar outline This seminar will consider the use of online assessment and feedback within the Learning Skills course on the LEAPS Summer School, a pre-university academic programme for school leavers in south-east Scotland. The session will explore how the practical application of assessment theory was used to confront challenges and adapt to changes within the learning environment. This will include a rationale for the use of quiz exercises with automated feedback, and exemplars and marking exercises, as part of a blended learning system. The session will also reflect on some of the difficulties experienced in putting theory into practice.

The seminar will begin with a ten-minute presentation followed by around forty minutes of discussion, with participants encouraged to critique the outlined system of assessment and feedback, as well as proposing alternative approaches.


Provides academic preparation to aid transition

2

Generally unfamiliar with formal online learning

Arrive straight from high school

Holding conditional offer of a university place

Varying levels of confidence and motivation

7-week pre-university programme

Taught at Year 1 degree level

Delivered on campus at Edinburgh universities

Offers extra opportunity to demonstrate ability

BACKGROUND AND CONTEXT


3

The course covers themes including academic writing, oral communication skills, critical thinking, information research skills and presenting evidence.

Assessment comprises a 1500-word individual essay and a team project that includes a 20-minute presentation of arguments and evidence.

The course promotes safe experimentation where students are encouraged to try out new learning strategies.

REFERENCES Boud, D. (2000). 'Sustainable assessment: rethinking assessment for the learning society.' Studies in Continuing Education, 22.2, pp. 151-167. Hounsell, D., Xu, R. and Tai, C.-M. (2007). Balancing Assessment of and Assessment for Learning. (Scottish Enhancement Themes: Guides to Integrative Assessment, nos. 2 & 3) Gloucester: QAA.

THE LEARNING SKILLS COURSE

PROMOTE LEARNING

Introduces students to learning strategies that will contribute towards a smooth academic transition into higher education and academic success in subsequent years of study.


4 RESPONDING TO A CHANGING ENVIRONMENT

REFERENCES Sharpe, R., Benfield, G., Roberts, G. and Francis,R. (2006). The undergraduate experience of blended e-learning: a review of UK literature and practice. The Higher Education Academy. Retrieved: 4 April 2011. http://www.heacademy.ac.uk/assets/York/documents/ourwork/research/literature_rev iews/blended_elearning_full_review.pdf

Our response was to shift the emphasis from an entirely campus-based course to a place where blended learning takes place, with materials hosted on a purpose-designed course site/virtual learning environment. This includes exemplars with marking activities, and quiz exercises with canned feedback.


5 Each week a quiz exercise is added to the VLE that revisits a key theme covered during class. In completing the quiz, students are encouraged to ‘write out’ against each of the possible answers before selecting what they feel is the strongest response. Students are aware that they are not being graded, in order to promote safe experimentation.

References: Boud, D. (1995). 'Assessment and learning: contradictory or complementary’. In: Knight, P. ed. Assessment for Learning in Higher Education. London: Kogan Page/SEDA. pp. 35-48. Draper, S. (2009). 'Catalytic assessment: understanding how MCQs can foster deep learning'. British Journal of Education Technology (special issue on e-assessment), 40(2) 285-293. Gibbs, G. and Simpson, C. (2004-5). 'Conditions under which assessment supports students' learning.' Learning and Teaching in Higher Education, 1, pp.3-31. Nicol, D. and Macfarlane-Dick, D. (2006). 'Formative assessment and self-regulated learning: a model and seven principles of good feedback practice', Studies in Higher Education, 31(2), 199-218.

ONLINE QUIZ EXERCISES AND CANNED FEEDBACK

Students received automated canned feedback that includes a commentary from the tutors against each of the possible responses, including what they feel to be the strongest of the options. Where students have selected a weaker response they are directed to a specific part of the lecture or other materials covered in class in order to take corrective action.


6

Nicol, D. (2007). 'Laying a foundation for lifelong learning: case studies of technology-supported assessment in large first year classes'. British Journal of Educational Technology. 38.4, pp. 668-678.

ONLINE QUIZ EXERCISES AND CANNED FEEDBACK

Collated responses sent to tutor

Canned feedback sent to student

Student uses corrective advice within canned feedback to address gaps in understanding and arrives at subsequent class with a better overall grasp of the topic.

STUDENT

TUTOR Tutor adapts subsequent class-based session to address identified gaps in understanding (or decides not to revisit topics where clear understanding is demonstrated).


7 Step 1: Exemplar presentation Students watch a video of a presentation delivered by undergraduate students. The presentation demonstrates good practice however isn’t flawless, in order to avoid intimidating students.

Step 2: Marking exercise Students download the course assessment criteria and then reflect on the quality of the viewed presentation using the mark sheet that will be used to judge their own work at the end of the course.

Students watch a ‘director’s cut’ of the presentation where tutors discuss the strengths and weaknesses of the work, before downloading the tutors’ completed assessment sheet and comparing it to their own equivalent.

Step 3: Marking exercise

REFERENCES Gibbs, G. and Simpson, C. (2004-5). 'Conditions under which assessment supports students' learning.' Learning and Teaching in Higher Education, 1, pp.3-31. Hounsell, D. and McCune, V. (2003). 'Students' experiences of learning to present'. In: Rust, C., ed. Improving Student Learning Theory and Practice – Ten Years On. (Proceedings of the Tenth International Symposium on Improving Student Learning, Brussels, September 2002. Oxford: CSLD. pp. 109-118. Keppel, M. and Carless, D. (2006). 'Learning-oriented assessment: a technology- based case study'. Assessment in Education, 13.2, pp. 179-191.

EXEMPLARS AND MARKING EXERCISES


FORMATIVE ASSESSMENT AND FEEDFORWARD 8

REFERENCES: Biggs, J. (2003). ‘Aligning teaching for constructing learning'. The Higher Education Academy. Retrieved: 4 April 2010. https://www.vle.ed.ac.uk/webct/RelativeResourceManager/Template/readings/Biggs_ 2003.pdf Chickering, A. and Gamson, Z. (1987). ‘Seven principles for good practice in undergraduate education’. American Association of Higher Education Bulletin vol.39 no.7 pp.3-7 Keppel, M. and Carless, D. (2006). 'Learning-oriented assessment: a technology- based case study'. Assessment in Education, 13.2, pp. 179-191.

FEEDFORWARD

ONLIN

E

IN C

LASS

Redesigning the course as a place of blended learning increased opportunities for formative assessment and feedback. Each part of the learning system is aligned, providing regular correctional advice and guidance that feed forward towards summative assessment.

Personal Response Systems (‘clickers’)


9 WHAT HAPPENED IN PRACTICE…

Only around a quarter of students completed the online quiz exercises.

Tutors received a false impression of the overall level of understanding within the class (in relation to the tested concepts).

The students who achieved the highest marks for the course were almost entirely those who made regular use of the quiz exercises.

The quality of coursework has improved since the introduction of the marking exercises.

Compared to before, it is clear that more students are aiming to satisfy the assessment criteria within their summative work.

Very few students challenge or query the marks and feedback provided after summative assessment.


1.  Course designers might find a solution to assessment dilemmas within the literature, however theory does not automatically translate into success in the classroom.

2.  Each assessment system is situated within its own unique environment, prompting a need to align assessment to other elements of the learning system, and the needs of the audience.

3.  Even careful planning cannot account for every eventuality on every occasion. In particular, it can be difficult to anticipate how the intended audience will respond to assessment (and how attitudes can vary across the audience).

4.  Using a combination of approaches can potentially be more effective than employing a single approach.

5.  It may take several stages of development before a satisfactory assessment approach is achieved (and even then, it may be necessary to be pragmatic in accepting a ‘best fit’ rather than an ideal solution.)

10 CONCLUSIONS

online assessment

Documents

learning skills course

integrative assessment

rethinking assessment

sustainable assessment

balancing assessment

use of online assessment

context online assessment

learning society