faculty of business & law “idemand feedback: investigation of feedback behaviour and...

Faculty of Business & Law

“iDemand feedback: investigation of feedback behaviour and preferences with efficient solutions for improving student

satisfaction.

Ollie Jones


Paper Structure

Element 1 – Feedback PreferencesElement 2 – i-demand Intervention


Element 1Student Perceptions

of Feedback


Context

Student Perceptions of feedback– Weaver (2006), Rae & Cochrane (2008)

Not engaged with Feedback– Winter & Dye (2005), Mutch (2003)

Grade Sensitive ?– Taras (2003) Bevan et al (2008) and Rowe

and Wood (2007)


Context – Sample Size 50 from 260


Which is the top preference for students?

1) Feedback on a practice activity closely related to the assessment task(s).

2) Feedback on my draft work for the assessment

3) Generic Feedback about an assignment on X-stream

4) My work scored in a Matrix showing what’s expected for each grade band

5) Handwritten comments on my assessment work

6) One-to-one face-to-face chat with the lecturer concerned after the assignment


Preference Scores based First Choice Only

0 2 4 6 8 10 12

A. Feedback in tutorials about my general work beforethe assessment.

B. Feedback on a practice activity closely related to theassessment task(s).

C. Feedback on my draft work for the assessment

D. Generic Feedback about an assignment on X-stream

E. My work scored in a Matrix showing what’s expectedfor each grade band

F. Handwritten comments on my assessment work

G. Handwritten summary sheets attached to myassessment work

H. Typed summary sheets about my assessment work

I. Audio file with detailed feedback about myassignment

J. Face-to-face debriefing about an assignment with thewhole group in a lecture/tutorial

K. One-to-one face-to-face chat with the lecturerconcerned after the assignment


0 2 4 6 8 10 12 14 16

Great value

Significant Value

Limited Value

No value

Un-helpful


0 2 4 6 8 10 12 14

Great value

Significant Value

Limited Value

No value

Un-helpful


0 2 4 6 8 10 12 14 16

Great value

Significant Value

Limited Value

No value

Un-helpful



Feedback Manifesto –H1

Not all students want the same approach to feedback, but on balance, we approach feedback as though they do.


I believe that feedback before the assignment [ formative ] is more valuable than after [summative]

A. Strongly AgreeB. AgreeC. Neither Agree nor disagreeD. DisagreeE. Strongly Disagree


0 5 10 15 20 25

Strongly Agree

Agree

Neither agree nordisagree

Disagree

Strongly Disagree

I believe that feedback before the assignment [ formative ] is more valuable than after[summative]


0 5 10 15 20 25

Strongly Agree

Agree


Disagree

Strongly Disagree

I would rather have feedback on my actual draft work for an assignment, than feedback on similar or related tasks or exercises on

the same topic.


Preference Scores based on 3,2,1 for First, Second Third

0 5 10 15 20 25 30 35 40 45 50

G. Handwritten summary sheets attached to myassessment work

E. My work scored in a Matrix showing what’s expectedfor each grade band


I. Audio file with detailed feedback about myassignment


J. Face-to-face debriefing about an assignment with thewhole group in a lecture/tutorial

H. Typed summary sheets about my assessment work

K. One-to-one face-to-face chat with the lecturerconcerned after the assignment

A. Feedback in tutorials about my general work beforethe assessment.

B. Feedback on a practice activity closely related to theassessment task(s).



Efficiencies

Bloxham & Boyd (2007)Denton et al (2008)Van den Bloom (2004)Hepplestone and Mather (2007)


I am more likely to obtain my feedback if it is in X-stream, rather than having to collect it.

A. Strongly AgreeB. AgreeC. Neither Agree nor disagreeD. DisagreeE. Strongly Agree


0 2 4 6 8 10 12 14

Strongly Agree

Agree


Disagree

Strongly Disagree

I am more likely to obtain my feedback if it is in X-stream, rather than having to collect it.


I am more interested in the mark than feedback for the final assignment, and will only seek detailed feedback if it differs from my expected grade.



0 2 4 6 8 10 12 14

Strongly Agree

Agree


Disagree

Strongly Disagree

I am more interested in the mark than feedback for the final assignment, and will only seek detailed feedback if it differs from my

expected grade.


Intervention i-demand feedback


Context

260 Students


Hierarchy of Summative feedback

Generic Feedback

Individual Rubric & Comments

DetailedIndividual

Resource High

3 weeks

2 weeks

4 week‘on demand’


How many students listened to generic feedback

A. 5-10%B. 10-20%C. 20-30%D. 30-50%E. 50%+


How many students accessed their feedback rubric?

A. 5-10%B. 10-20%C. 30-40%D. 40-50%E. 50%+


0

20

40

60

80

100

120

140

160

180

1.Very Likely 2.Quite Likely 3.Not sure 4.Quite Unlikely 5.Very Unlikely

How likley is is that you will want individual audio feedback

Series1

How likely is that you will want individual audio feedback?


How many students actually requested individual audio feedback?

A. 5-10%B. 10-25%C. 25-40%D. 40-50%E. 50%+


How many students [of those who requested it ] listened to their feedback file?

A. 5-20%B. 20-40%C. 40-50%D. 50-70%E. 70%+


Le grand finale?


Hierarchy of Summative feedback

Generic Feedback

Individual Rubric & Comments

DetailedIndividual

Resource High

3 weeks

2 weeks

4 week‘on demand’


“These lecturers believe students are more likely to collect, read and act on feedback if it is on formative assessments, such as comments on essay drafts. Even the exemplary lecturers experience that feedback on summative assessments is most likely to be left uncollected at the end of the semester”…(Jollands et al 2009)


If the mark is what I expected then I am usually satisfied with simple feedback using a scored matrix/rubric on my assignment ( such as those used in ops mgmt in S1)



0 5 10 15 20 25

Strongly Agree

Agree


Disagree

Strongly Disagree

If the mark is what I expected then I am usually satisfied with simple feedback using a scored matrix/rubric on my assignment ( such as those

used in ops mgmt in S1)


0 2 4 6 8 10 12 14 16 18

Great value

Significant Value

Limited Value

No value

Un-helpful

E. My work scored in a Matrix showing what’s expected for each grade band


The tricky 10%;

Who are they?


0%

5%

10%

15%

20%

25%

30%

35%

FAIL 40 50 60 70

Feedback Request Versus Final Grade

Series1


Self vs Actual assessment scores against ACtual Grade

-40

-30

-20

-10

0

10

20

30

40

50

1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153 161 169 177 185 193 201 209 217 225

Series1

Linear (Series1)


Feedback Request % in Difference bands

0% 10% 20% 30% 40% 50% 60% 70%

30-40

20-30

10-20

-0+10

0-10

-10-20

Scored Rubric

Detailed AudioFeedback


0 2 4 6 8 10 12 14

Strongly Agree

Agree


Disagree

Strongly Disagree

I am more interested in the mark than feedback for the final assignment, and will only seek detailed feedback if it differs from my

expected grade.


Reasons for seeking additional feedback

0 1 2 3 4 5 6 7 8 9 10

To understand my mark

I want to learn from this assignment tohelp with ones in other subjects

Just interested

To justify my mark

I was not happy with my Mark

The other feedback was not as specific

I love feedbackon my ownperformance

I couldn’t relate the other feedback tomy work

I didn’t undetand the other feedback

because it seemed an easy way to getfeedback


Feedback Manifestos Providing ‘in depth’ or ‘detailed’ summative

feedback for all students after an assessment is neither effective ,efficient or desirable.

Detailed feedback could be offered to those students who want (need) it

Focus ( and hence resource )could be mostly shifted from summative to formative feedback

Compare the tutorial activity to the top three feedback preferences?

Consider the equity issue raised by Taras (2006) in the nature of formative feedback and Sadler (2010) comments about standards and Gibbs & Simpson (2009) about peer assessment

faculty of business & law “idemand feedback: investigation of feedback behaviour and...

Documents

faculty of business

demand slide

assignment slide

detailed feedback

feedback rubric

feedback preferences

demand intervention

feedback manifesto h1