Frances ChetwyndChris Dobbyn

The Open University. UK

Consistency v

Autonomy:

effective feedback

to a very large

cohort

13/04/2012HEA Stem Conference 2012

Introduction

Project: Investigating major changes to assessment and feedback on a new module

DiscussionWhy make changes?What changes were made?Views on changes?

13/04/2012HEA Stem Conference 2012

TU100: My Digital Life

60 credits with extremely varied teaching material

4500 students and 200 tutors

Poor retention and progression levels in predecessor courses

13/04/2012HEA Stem Conference 2012

4,500 different combinations

13/04/2012HEA Stem Conference 2012

Image: Ralph Greenwell

Sense Board with a specially-designed language, Sense

Challenges

Build skills for future, self-regulated learning

Create “joined-up” assessment

Improve retention and progression

Change a tutor culture deeply wedded to marks

Preserve consistency at the same time as promoting tutor autonomy

13/04/2012HEA Stem Conference 2012

Earlier research findingsChetwynd & Dobbyn (2011)

reported problem in the areas of: Assignment questions Marking guidance Tutor feedback practices

Motivation of Marking Guides was consistency of marking over feedback

Effective feedback essential for building independent, self-regulated learners (Nicol 2008)

13/04/2012HEA Stem Conference 2012

New feedback taxonomy

13/04/2012HEA Stem Conference 2012

Chetwynd & Dobbyn (2011)

A new project

Research project based on TU100 My Digital Life

New assessment materials and assessment style

New approach to tutor advice

Evaluation

Identify “choke points” in the course as potential targets for remedial material

13/04/2012HEA Stem Conference 2012

New assessment materials

Explicit testing of skills

e.g. “List the instructions your program will step through in order to parse the RSS feed…”

e.g. “To help you structure your answer, sample reports are given as answers to Block 2 Part 3 Activity 29 and Block 2 Part 5 Activity 18.”

13/04/2012HEA Stem Conference 2012

New assessment materials

Explicit concentration on skills

80% Assignment questions: 20% skills mark

English language usage

Programming skills

Structure/layout

Numeracy (scientific notation; units)

Referencing

13/04/2012HEA Stem Conference 2012

New Tutor Guides

Emphasis on feedback rather than marks

New content Explicit feedback advice Separation of retrospective and

future-altering feedback advice

New structure Separation of skills and content

feedback advice

Repetitive guidance removed

Granularity of marks reduced

13/04/2012HEA Stem Conference 2012

Evaluation

1. Selected group of tutors; semi-structured interviews

Before first assignmentHalf way through moduleAfter EMA marking

2. Monitoring reports

3. End of presentation survey

13/04/2012HEA Stem Conference 2012

Preliminary results from interviews

Varying degrees of enthusiasm!All liked – focus on skills;

future-altering adviceSome liked – marks

granularity; new structureSome disliked – greater

freedom; separation of skills

13/04/2012HEA Stem Conference 2012

Preliminary results from interviews

“Has made me think about things differently…”

“…Not completely confident about how to tackle

skills aspects…”

“I could see the flow of questions and where they were going …”

“…Sometimes difficult to make retrospective feedback more

positive”13/04/2012HEA Stem Conference 2012

13/04/2012HEA Stem Conference 2012

References

Brown, E. and Glover, C. (2006) Evaluating written feedback. In Innovative Assessment in Higher Education C. Bryan & K. Clegg 81-91. Abingdon: Routledge.

Crisp, B. (2007). Is it worth the effort? How feedback influences students’ subsequent submission of assessable work. Assessment and Evaluation in Higher Education 32(5), pp. 571-581.

Chetwynd, F. D. & Dobbyn, C. (2011): Assessment, feedback and marking guides in distance education, Open Learning: The Journal of Open, Distance and e-Learning, 26(1), pp. 67-78.

Gibbs, G. (2010) Does assessment in open learning support students? Open Learning: The Journal of Open and Distance Learning, 25(2), pp. 163 — 166.

Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students’ learning, Learning and Teaching in Higher Education, 1, pp. 3-31.

Hounsell, D. (2007) Towards more sustainable feedback. In Rethinking Assessment in Higher Education D. Boud & N. Falchikov (Eds) 101-113 Abingdon: Routledge.

Lea, M. R. & Stierer, B. V. eds. (2000) Student writing in higher education, new contexts, Maidenhead: Open University Press.

Lea, M. R. & Street, B. V. (1998) Student writing in higher education: An academic literacies approach, Studies in Higher Education, 23(2), pp. 157-172.

Nicol, D. (2008) Transforming assessment and feedback: Enhancing integration and empowerment in the first year. Quality Assurance Agency, Scotland.

Nicol, D. & Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: a model and seven principles of good feedback practice, Studies in Higher Education, 31(2), pp. 199-218.

Price, M., Carroll, J., O’Donovan, B. & Rust, C. (2011) If I was going there I wouldn’t start from here: a critical commentary on current assessment practice, Assessment & Evaluation in Higher Education, 36(4), pp. 479-492.

Sadler, D. R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18(1), pp. 19-144.

Walker, M. (2009) An investigation into written comments on assignments: do students find them usable? Assessment & Evaluation in Higher Education, 34(1), pp. 67-78.

Weaver, R. (2006) Do students value feedback? Student perceptions of tutors’ written comments. Assessment and Evaluation in Higher Education, 31(3), pp. 379-394.