assessment, feedback and achievement · • assessment feedback is the least satisfactory aspect of...

Assessment, Feedback and Achievement: Exploring approaches to ‘sustainable feedback’.

Chris Beaumont,

Faculty Senior Learning & Teaching Fellow; Head of Department of Computing, Edge Hill University

[email protected]

mailto:[email protected]

http://www.edgehill.ac.uk/Faculties/FAS/Business/

To explore:

• Research-informed models of good practice in

Assessment Feedback.

• Problems with feedback.

• Possible solutions: ‘Sustainable Feedback’

2

Aims

http://www.edgehill.ac.uk/Faculties/FAS/Business/ 3

Why are we bothered?

Student: • assessment is a major influence on learning (Biggs, 2003; Boud, 2007)

• is the most powerful single influence on student achievement (Hattie,

1987; Black and Wiliam, 1998; 2003);

• assessment feedback is the least satisfactory aspect of student

experience:

– lowest satisfaction score in National Student Satisfaction surveys

(NSS) since 2005.

Staff: • ‘action without feedback is completely unproductive for the learner’

(Laurillard, 2002 p.55);

– QAA subject reviews identify assessment as one of weakest features

(Rust, 2005).

– can perceive their feedback to be more valuable than students do (Carless,

2006); (Maclellan, 2001);


So, What is Feedback?

‘Hopefully useful information’

(Boud, 2013, p.203)


So, What is Quality Feedback?

‘Assessment involves identifying appropriate standards

and criteria and making judgements about quality’

(Boud, 2000, p.151)

Activity:

• Write down 3 criteria for quality feedback – one per post-it.

• Swap your post-its with a neighbour.

• Be prepared to feedback.


So, What is Quality Feedback?

Acted Upon

Received and Attended To

Understandable

(to Students)

Linked to Criteria

and Purpose

Focus on Learning

Timely

Sufficiency Frequency and Detail

Quality Feedback

Conditions Under Which Assessment Supports Students’ Learning

Gibbs and Simpson (2004-5)


1. Helps clarify what good performance is (goals, criteria, expected standards).

2. Facilitates the development of self-assessment (reflection) in learning.

3. Delivers high quality information to students about their learning.

4. Encourages teacher and peer dialogue around learning.

5. Encourages positive motivational beliefs and self-esteem.

6. Provides opportunities to close the gap between current and desired performance.

7. Provides information to teachers that can be used to help shape the teaching.

7

7 Principles of Good Feedback

Practice

Nicol and Macfarlane-Dick (2004)


Feedback in FE:

the Dialogic feedback Cycle

DfC (FE) 1.

PREPARATORY

GUIDANCE

Explanation of criteria

Discussion of task

Exemplars

Model answers

Target grades

2.

IN-TASK

GUIDANCE

Drafts and Practice

Generic feedback

(pre- assessment tips)

Peer assessment

3.

PERFORMANCE

FEEDBACK

Written

Discussion

Formative - timely

Standards related

DISTRIBUTE

ASSIGNMENT

REVIEW

FEEDBACK

SUBMIT

ASSIGNMENT

Staff and Students’ prior

experience of assessment

Action points

for feedforward

8


9

Target Grades… ‘It’s .. taken from your GCSEs and its like the lowest grade you are expected to get….you often have to do it again until its either above the target grade or on your target grade..’ [FE Student]

HE students’ Prior Experience

‘..when I did my A levels we knew

exactly what they wanted from us..’ [HE Student] ‘…….they do it more when you are planning an essay.. examples of what you have to put in …but you have to do it though..’ [School Student]

‘..they are getting marked according to the exam scheme all the way through and eventually it sinks in, we absolutely hammer [the exam criteria] , the main thing..’ [School Tutor]


10

In-task guidance

‘we’ve learned to rely on drafts …so if you’re not getting that at University, it’s going to be a big shock.’ ‘..we could hand coursework in as many times as we wanted.’ ‘..we get our coursework given to us in like tasks..’ [FE Students]

‘ we do drafts in coursework..that’s actually part of what they are supposed to do..’ ‘...there is ongoing communication of feedback..’ [FE Tutors]


11

Feedback on performance

‘...written down and then the teacher would sit and talk it through with you. [HE Student]

‘..they give you suggestions and try and guide you… ‘..usually within a week..’ [FE Students]

‘that’s the policy of my team….we say, we’ll return it in one week..’ [FE Tutors] ‘its not….enough to put on ‘good essay'.. because for us assessment objectives are absolutely crucial…..so our comments are geared towards specific assessment objectives and how to improve on them.’ [FE Tutors]


12

Support

‘..even if we have not got a lesson, we can always go and find them for help or we have support lessons [FE Student]

‘….I felt really supported… I could go to anyone and ask for help ….’ [FE Student]

‘.it’s the pressure of you really want the students to do well..’

‘we’re under so much pressure to get exam results and for the student to achieve and we tend to spoon feed them as much as we can ..’ [FE Tutors]


HE students’ Prior Experience

Focus Groups: findings consistent across disciplines and institutions.

Reported high satisfaction levels with School/FE experience:

Support

- explicit pre-assessment guidance

- accessibility to tutors: personal, usually not online

- explicit feedback linked to grade improvement: motivation

Frequent feedback

- formative: (65%*) drafts for coursework/ exam practice

- timely: (74%*), usually 1 week turnaround

- detailed: exemplars

- understandable: written and oral feedback

Awareness of criteria/marking scheme

- some peer assessment

Read and acted on (80%*): culture of target grades

Higher

Education

Perceptions of Quality Feedback (norms)

* Sample size 180


• There is a ‘right answer’ (c.f. Perry)

• The teacher’s role is to make things

simple.

• Learning is about memorisation.

• Discussion with peers is confusing and time

wasting.

• Assessment is about measurement

(Price, 2012)

14

Students’ Expectations of Assessment in HE


Feedback in FE – the solution?

Dialogic feedback Cycle DfC (FE)

•Delivers improvements in grades •Does Not promote ‘independent learning’

‘.. we were spoon fed practically at what we were doing.’

1.

PREPARATORY

GUIDANCE

Explanation of criteria

Discussion of task

Exemplars

Model answers

Target grades

2.

IN-TASK

GUIDANCE

Drafts and Practice

Generic feedback

(pre- assessment tips)

Peer assessment

3.

PERFORMANCE

FEEDBACK

Written

Discussion

Formative - timely

Standards related

DISTRIBUTE

ASSIGNMENT

REVIEW

FEEDBACK

SUBMIT

ASSIGNMENT



Action points

for feedforward


Culture gap

• “A key characteristic of UK higher education is the emphasis placed on the responsibility of students for their learning.”

(QAA, 2012)


ACTIVITY

Feedback

to group

In your groups discuss the issues

with feedback

Nominate a group member to report back


• Giving appropriate feedback that is useful at that point to that individual student. Inherent difficulty of

attempting to change student concepts/ behaviour.

• Getting students to act on feedback so there is a

positive impact on learning.

• Students’ prior experience/ perceptions

• Burden of time….

18

Some issues


Dealing with the problem

1. Scaffold students into academic community of practice through ‘front loading’ (Rust, 2001) - feedback as preparatory guidance.

2. Explicitly teach Self-Regulated Learning skills to help transition, taking account of students’ prior assessment experiences.

3. Assessment protocols to encourage student engagement with formative feedback to maximise ‘feedforward’.

4. Feedback as a dialogue, not a monologue

Reconceptualise feedback as a guidance process for self-regulated (independent) learning:


Recommendations

• Induction – the first year

– Concepts of assessment & feedback in university.

– Connection of Assessment and Learning.

– Familiarity with the techniques of assessment & feedback.

– Understanding the assessment and feedback systems of university.

– Develop an understanding of the quality standards (what is good).

• Importance of early formative assessment opportunities

– Within the first 3 weeks of semester 1 @ Oxford Brookes (Price,2012)


Sustainable feedback?

(Imgfave.com,2012)


Sadler

• Sustainability is about developing self-assessment capability.

• For many students, feedback seems to have little or no impact.

• They also need to identify, with near certainty, the particular aspects of their work that need attention.

• For these to occur, students must possess critical background knowledge. They must appropriate for themselves three fundamental concepts:

• Task compliance (answer the question)

• Quality (holistic notion of what is good)

• Criteria (A property useful in determining quality)

(Sadler, 2010)


Burke

• Getting students to engage with feedback

• Preparing students for your feedback

– Feedback proforma exemplars with explanation

• Student response to feedback

– Analysis of feedback

– Identify what you understand, don’t and identify points to discuss with tutor

• Tutorial

– Create action plan (Burke, 2011)

• Burke,D. (2011) “Now I’ve got the feedback, what do I do with it?” Strategies For Students to Get More Out of Tutor Feedback. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education Vol. 6, No. 1, April 2011, pp. 43-60


Smart feedback

• When is time spent on marking least effective?

– When comments will not be considered

– When the course is inadvertently designed to undermine feedback.

– When students won’t be using useful information subsequently

• typically at the end of a course/semester

– When grades are seen more important than learning • this normally means our grading practices are not supporting

learning

(Boud, 2011)


Smart feedback

• Machines – Simple repetitive self-practice – Lots of practice needed

• Peers – Where communication to others is important – When peers have been trained – To foster co-operation – When both parties learn

• Tutors – Where other students have misconceptions which must be avoided – When tutors need to know to change their own action – Where certification is important

• Others – When they have unique valued knowledge (e.g. employers)

Boud (2011)


The DFC adapted for HE

26

Df C (HE) 1.

PREPARATORY

GUIDANCE

Audit: prior experience

Assessment Criteria

Workshops

Exemplars/Models

Discussion

2.

IN-TASK

GUIDANCE

Differentiated, in-class

guidance activities

Drafts and Practice

Generic feedback

Peer Mentoring

Referral

3.

PERFORMANCE

FEEDBACK

Written & Discussion

Self-assessment

Protocols for

feedforward /

feedback

DISTRIBUTE

ASSIGNMENT

REVIEW

FEEDBACK

SUBMIT

ASSIGNMENT



Action points

for feedforward


Spiral Curriculum

27

Scaffolding Self Regulated

Learning

Sustainable assessment for future learning using the Df C (HE)

….fair enough, we shouldn’t be spoon fed in uni , but at least we should have a spoon…. [First year HE student]

1.

PREPARATORY

GUIDANCE

Audit of

prior experience

2.

IN - TASK

GUIDANCE

Differentiated

guidance activities eg

patchwork text Peer assessment

of exemplars

3.

PERFORMANCE

FEEDBACK

Begin

Self - assessment

( eg . checklist )

DISTRIBUTE

ASSIGNMENT

REVIEW

FEEDBACK

SUBMIT

ASSIGNMENT

Staff and Students’ prior experience of assessment

Action points for feedforward

1. PREPARATORY

GUIDANCE

Self - Regulatedd

2. IN - TASK

GUIDANCE

Self - Regulated

3. PERFORMANCE

FEEDBACK Includes

Self - assessment (with grade )

Peer feedback

DISTRIBUTE

ASSIGNMENT

REVIEW

FEEDBACK

SUBMIT

ASSIGNMENT



Action points

for feedforward


Feedback for Future Learning?

• First year students expect feedback as a formative, guidance process: the Dialogic feedback Cycle – in practice does not develop independent learning.

• HE culture expects self-regulated learning – no systematic, integrated feedback guidance in HE

• Feedback re-conceptualised as a process using the DFC, with activities adapted to scaffold SRL – scaffold student autonomy for sustainable feedback.

– Refocus effort on early formative work, when students are most ‘malleable’.


Feedback by Design

• Feedback should be designed into the course:

• Course organised so that students can act on the

feedback so it is no longer just ‘hopefully useful

information’ (activities to use it).

• Relevant high quality information at the point students

can use it. (What/ When/ Who)

• Students must understand it (Dialogue)

• 100% pass rate!


Discuss

30

31

References

Alexander, R. (2004) Towards Dialogic Teaching: rethinking classroom talk. 3rd. ed. University of Cambridge: Dialogos. Beaumont,C., O’Doherty, M. and Shannon, L., 2011. Reconceptualising assessment feedback: a key to improving student learning?, Studies in Higher Education, 36(6), 671-687. Biggs, J. (2003) Teaching for Quality Learning at University. 2nd ed. Buckingham: Society for Research into Higher Education and Open University Press. Black, P., Harrison, C., Marshall, L. and Wiliam, D. (2003). Assessment for Learning: putting it into practice. Maidenhead: Open University Press. Black, P. and Wiliam, D. (1998) Assessment and classroom learning. Assessment in Higher Education, 5 (1), pp.7-74. Boud, D. (2011) Feedback for Learning, What are we missing? Boud, D. (2007) Reframing assessment as if learning were important, In D. Boud and N. Falchikov (eds) Rethinking Assessment in Higher Education: learning for the longer term. London: Routledge. Boud, D. and Falchikov (2007) Developing assessment for informing judgement, In D. Boud and N. Falchikov (eds) Rethinking Assessment in Higher Education: learning for the longer term. London: Routledge. Boud, D. and Molloy,E. (eds) (2013) Feedback in Higher and Professional Education: understanding it and doing it well, Routledge Braun, V. and Clarke, K (eds) (2006) Differing perceptions in the feedback process. Studies in Higher Education , 31, (2) pp.219-233. Burke,D. (2011) “Now I’ve got the feedback, what do I do with it?” Strategies For Students to Get More Out of Tutor Feedback. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education Vol. 6, No. 1, April 2011, pp. 43-60 Forbes, D., & Spence, J. (1991). An experiment in assessment for a large class. In R.Smith (Ed.), Innovations in engineering education. London: Ellis Horwood. Gibbs, G. (2006) Why assessment is changing , In C. Bryan and K. Clegg (eds) Innovative Assessment in Higher Education London: Routledge Gibbs, G. & Simpson, C. (2004-5) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, 1 (1), pp. 3-31. Hattie, J.A. (1987) Identifying the salient facets of a model of student learning: a synthesis of meta-analyses. International Journal of Educational Research, 11, pp.187-212. Higgins, R., Hartley, P. & Skelton, A. (2002) The conscientious consumer: reconsidering the role of assessment feedback in student learning, Studies in Higher Education, 27(1), pp.53-64. Hounsell, D., McClune, V., Hounsell, J. and Litjens, J. (2008) The quality of guidance and feedback to students, Higher Education Research and Development, 27 (1) pp. 55-67. Hounsell, D (2007) Towards more sustainable feedback to students, In D. Boud and N. Falchikov (eds) Rethinking Assessment in Higher Education: learning for the longer term. London: Routledge. National Student Survey (2007) Teaching Quality Information Data. Available at: http://www.hfce.ac.uk/learning/nss/data/2007 [Last accessed 16 November 2007] see also: (NSS ,2006) http://www.hefc.ac.uk/pubs/rdreports/2007/rd14 07/rd14 07aum.doc and NSS (2005) http://www.hefce.ac.uk/pubs/reports/2006/rd22 06/rd22 06.doc [Last accessed 22 October 2007]. Laurillard, D (2002) Rethinking University Teaching: a conversational framework for the effective use of learning technologies. 2nd ed. London: Routledge. Nicol, D.J. & Macfarlane-Dick, D. (2004-5) Rethinking formative assessment in HE: a theoretical model and seven principles of good feedback practice. In C. Juwah, D. Macfarlane-Dick, B. Matthew, D. Nicol, D. & Smith, B. (2004) Enhancing student learning though effective formative feedback, York: The Higher Education Academy. Available at: http://www.heacademy.ac.uk/assessment/ASS051D_SENLEF_model.doc. [Last Accessed 20 February 2006]. Ramsden, P (2003) Learning to Teach in Higher Education, 2nd. rev. ed. London: Routledge. Rust, C. (2005) A social constructivist assessment process model: how the research literature shows us this could be best practice, Assessment & Evaluation in HE, 30 (3) pp 231-240. Rust, . A (2001) A briefing on Assessment of Large Groups. Series no. 12 York: Generic LTSN Centre. Available at: http://www.swap.ac.uk/docs/ltsn/assess/12largegroups.pdf [Last accessed 20 October 2007]. Sadler, R. (1998) Formative assessment: revisiting the territory. Assessment in Education, 5 (1) pp.77-84. Sadler, R (1989) Formative assessment and the design of instructional systems, Instructional Science, 18, pp. 119-144. Sadler R. (2010): Beyond feedback: developing student capability in complex appraisal, Assessment & Evaluation in Higher Education, 35:5, 535-550 Strevyn, K., Dochy, F and Janssens, S,. (2005) Students’ perceptions about evaluation and assessment in higher education: a review, Assessment and Evaluation in Higher Education, 30(4), pp. 325-341.

http://www.hfce.ac.uk/learning/nss/data/2007

http://www.hefc.ac.uk/pubs/rdreports/2007/rd14 07/rd14 07aum.doc

http://www.hefce.ac.uk/pubs/reports/2006/rd22 06/rd22 06.doc

http://www.hefce.ac.uk/pubs/reports/2006/rd22 06/rd22 06.doc

http://www.heacademy.ac.uk/assessment/ASS051D_SENLEF_model.doc

http://www.swap.ac.uk/docs/ltsn/assess/12largegroups.pdf


Case Study: Computing

32

Dialogic

Feedforward

Cycle

D f C (HE)

Semester 1

Week

Spiral Curriculum:

Assessment activity

20-30 minutes at the start

of each weekly seminar

Nicol & Macfarlane-

Dick (2006)

Principles [1-7]

Team

Meeting

Gibbs and

Simpson

(2004-5)

Conditions for

student

learning

Pre -Guidance

1 - 6 Audit

of prior experience

Introduction to Core Criteria

Workshops

Discussion using exemplars

1

4

5

7

Feedback:

•Timely

•Linked to

Criteria

•Sufficiency

•Understand

•Focus on

Learning

•Attended To

•Acted Upon

In-task

Guidance

7 Discussion of assignment

criteria

Peer marking of exemplars

1

2

8 - 9 Peer/Tutor discussion of

students’ drafts

3

6

Submission

Feedback

for

Feedforward

10 Self marking (before mark

given)

2

11 -12 Tutorial feedback sessions

– discussion of written

comments with tutor

3

(Whitfield and O’Doherty et al, 2008)

assessment, feedback and achievement · • assessment feedback is the least satisfactory aspect of...

Documents