Exploring and utilising students' perspectives on feedback: a mixed

method, longitudinal approach

Simon Croker, Kara Peterson, Dr Peter Hills and Dr Rachel Manning

Importance of Feedback

• Where students make mistakes.• Where students can improve.• What students have done well.

NSS 2012 Results

Background Literature

• Hulme and Forshaw (2009) – form and content of feedback not helpful.

• Gibbs and Simpson (2004) – lecturers and feedback.

• Higgins, Hartley and Skelton (2001) – differing views on feedback meaning.

• Nicol (2010) – what feedback should be.• Duncan (2007) – little improvement.

Project Timeline

March – May 2013Data Collection Phase 1

February 2013Pilot Phase

January 2013

Beginning of Research

January – March 2014Evaluation of Project

November –December 2013

Focus Group Phase 2

June – October 2013Feeding Forward on

Perceptions of Feedback

Focus Group Participants

• Total of 41 participants.• Psychology students across the three

undergraduate years. • Range of academic profiles.• £9 or 1.5 credits.• Difficulty in recruiting participants.

Survey Participants

• Total of 98 participants• Psychology students from the three

undergraduate years and masters.• Range of academic profiles.• £6 or 1 credit.

Transcription and Analysis

• Audio recordings were transcribed.• Will be analysed using full thematic analysis. • Currently preliminary thematic ideas are

presented here. • Possible issues with current feedback system

and suggestions to improve.

Survey Data

Figure 1. Satisfaction levels with different aspects of feedback

Convenience - Issues

– ‘ I just read the e-mail. I haven’t collected any of mine (laughter). I just use the e-mails and go from there [INT: Yeah]. (2) I think it should be easier to actually collect your essay.’

– ‘No I’m not a fan of the codes. It’s just too many of those, [INT: um hm] I don’t remember them from my head so I always like look back what does B12 mean, oh B12, is doesn’t say here so I don’t even know what B12 means [INT: um hm].’

Convenience - Issues

Figure 2. Percentage of students that collect feedback and how they use feedback.

Convenience - Suggestions• Electronic Feedback– ‘Electronic could be easier to access as well cause you’ve

got like, I’m not particularly organized and we’ve got ridiculous amounts of paper work and trying to find the last times feedback to try and improve this time is just borderline impossible in my (laughter) lack of organizational skills…’

• Online sign up system for office hours.• Use of VLE could be improved.

Continuing Dialogue - Issues

– ‘ You basically get the feedback but you can’t, I’m always struggling to understand what it means in regards to my own coursework so yeah, that’s, that’s what I think.’

– ‘I like took that extra kind of time to make sure it was prefect but I still kept getting the same thing and I was just like are they really reading over it or is it, is that just like a generic thing.’

Continuing Dialogue - Issues― ‘Yeah I think it should be a lot more one on one cause like you said

some people aren’t very happy with the grades they’ve got and they don’t want it to be you know they might be embarrassed in front of other people talking about it so and I think erm the B1 and all that, I think that’s it’s helpful in a way but at the same time it just tells you what you’ve done wrong not where you can improve and that is a really big aspect that you should have to kind of do better in the future because there are some people who don’t go to see the lecturers erm about their grades and stuff and then they keep making the same mistakes over and over again so [INT: um hm]’

― ‘I don’t know whether it’s my slackness or whatever, but I don’t know who I’m, not allowed to speak to, but like, but I would never realised I could go to my personal tutor because they haven’t marked, well I don’t think they’ve marked it. So I would assume it wouldn’t be their area for me to go speak to them about.’

Continuing Dialogue - Issues

Figure 3. Percentage of students and their attitude to feedback.

Continuing Dialogue - Suggestions

• Utilising personal tutor system to create a continual dialogue with students.

• Electronic storage of feedback could also allow personal tutors to quickly identify students key areas that need improvement and offer support in improving these areas.

• Convert the process of feedback into an on going and continual process.

Consistency - Issues

– ‘It’s been quite mixed, like you either get three pages full of things that you could’ve done better or you get a little blank box and something and sort of a few little hand annotations on the thing…’

– ‘I think it should be standardised but obviously because of the topic it might vary what comments we’ll get written but I think if there was like a standard procedure for each topic then it might be a bit clearer, okay but overall I think it’s pretty good…’

Consistency - Issues

Figure 4. Student preferences for the structure of feedback.

Consistency - Solutions

• Bullet points highlighting the things that need improvement as well as the positive aspects of the work.

• Annotated comments throughout the work rather than the use of codes. – ‘I think personally I’d want like erm bullet points at the

start and then I’d want erm like details throughout, I don’t think I’d want, maybe some codes like about spelling or something like that or I don’t know a paragraphing or something but I really like the detail cause that’s what really helps me understand it and also you have to keep going back to check the different codes which takes ages.’

Consistency - Solutions

• Simpler coding system to highlight basic feedback ideas such

as spelling or referencing.– ‘I think that would be better with codes then having like

overall, like say missing things out, like more complicated things not using codes, more simple things using codes [Yeah] so referencing is quite a common one as well so codes would be better for that. ‘

Communication - Issues

• Many of the issues identified by students with the current system of feedback relate to the meta-concept of communication.

• Feedback could be greatly improved through a greater and more accessible communication channel between students and lecturers as well as among lecturers.

• All previous suggestions would require a good communication system to be implemented successfully.

Communication - Suggestions

• An electronic feedback system would increase students’ accessibility to feedback and would promote a greater utilisation of feedback.

• An increase in online resources made available to students would increase contact with lecturers.

• Personal tutor system to establish a continuing dialogue between students and staff, providing students with more feedback support.

• Consistency could be improved through an increased communication among lecturers and the implementation of a consistent structure of feedback.

Differences Between the Year Groups

F(3, 88) = 4.77, MSE = 245.88, p = .004. Figure 5. First year students perceive the feedback to be more useful than the grade, whereas the reverse is true for all other years,

Differences Between the Year Groups

Figure 6. Masters students are more likely to speak to a lecturer when they are confused with feedback than all other year groups, F(3, 85) = 3.75, MSE = 18.56, p = .014.

Correlations with Grades

• Students with lower grades were:– Marginally less satisfied, r(91) = .19, p = .066.– Less satisfied with comments highlighting what they had done well

r(91) = .18, p = .081.– Felt feedback was less helpful for use in future work, r(90) = .21, p

= .046.– Less likely to collect their feedback, r(88) = .25, p = .019.– Less likely to speak to lecturer, r(87) = .22, p = .035.– Feedback matters less to them, r(89) = .20, p = .058.

Correlations with Grades

• Students with higher grades compared to students with lower grades:– Felt that handwritten comments over typed comments showed that

lecturers were more interested in their work, β = .01, p = .011.– Preferred annotations over codes, β = .01, p = .001.– Preferred to speak to lecturers about feedback rather than using

codes, β = .02, p < .001 or reading paragraphs, β = .01, p = .007.

Correlations with Grades

• 66% of students wanted to receive all the comments on their work.

• 1/3 of students do not want all of their mistakes highlighted. • Out of the students who wanted only a certain number

highlighted, the mean number was 5 (range 2-10). • Students with higher grades wanted all the comments on

their work highlighted compared to students with lower grades wanted only a selected number of comments highlighted, β = .01, p = .004.

Recommendations

• Online submission of coursework, online marking and online returning of work.

• Codes input into the online system should be written out in full on the script.

• Hyperlinks from these codes to online resources that will help improve the identified issue.

• Accessible to both the student and their personal tutor.

Recommendations

• More user friendly VLE with ‘social media’ type activities. • Importance of VLE highlighted and made more accessible to

students. • Online booking system for office hours. • Personal tutor to mark all work in first year to make process

more transparent and less depersonalised. • Coversheets ask students to answer; what they want feedback

to cover, their anticipated grade, detail of feedback required and amount of work they put into assignment.

• Feedback consistent across lecturers in content and style.

Recommendations

• Positive elements of feedback. • Methods for improving. • Structured simply using bullet points. • Relate to mark scheme to highlight how grade was

constructed. • Inform student oh how to get a better mark.• Consistent codes across all forms of work. • Codes should be simpler and more specific. • Work containing written annotations rather than codes.

Recommendations

• Feedback workshops with all students in first year as part of a module.

• Increased use of VLE.• More concise module guides.• Further communication channels between staff and students. • Better communication between staff to ensure consistent

information and feedback.

Conclusion

• Focus groups and interviews conducted have already begun to provide a greater understanding of students perspectives of feedback.

• The findings suggest that there are some areas of the current feedback system that could use improvement.

• The implementation of the suggested changes, informed by a further analysis of the data, aims to put the increased understanding of students perspectives on feedback to use by creating an enhanced feedback experience.

References

Duncan, N. (2007). "Feedforward": Improving students' use of tutors' comments. Assessment and Evaluation in Higher Education, 32, 271 - 283.

Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, 1, 3 - 31.

Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in higher education, 6(2), 269-274.

Hulme, J., & Forshaw, M. (2009). Effectiveness of feedback provision for undergraduate psychology students. Psychology Learning & Teaching, 8(1), 34-38.

Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35, 501 - 517.