1. interACT: InteractiveAssessment andCollaboration viaTechnology

2. Outline Background to project What we did Where we are up to Preliminary evaluation data Challenges and futuredevelopment 3. The Project TeamRola Ajjawi Karen BartonGrant MurraySusie SchofieldProject Director Project OfficerLearning Technologist Project ManagerNatalie LaffertySean McAleerDavid Walkere-learning Advisor Assessment AdvisorLearning Technology Advisor 4. Recognition of a need for change. External examiner reports (2006-2011) End of course evaluations (2006-2011) Additional evaluation surveys conducted in 2010and 2011 as part of curriculum review HEA postgraduate student experience survey Staff discussions Feedback audit (5% random sample of allassignments submitted in 2011 audited)Outlined that 5. Specific issues raised:1. Inconsistency in the quality and quantity of feedback provided2. Assessment design (e.g. over-assessment, over-reliance on essays, lack of formative assessment)3. Timeliness of the feedback4. Lack of assessment and feedback dialogue 6. A critque of monologic feedback1-6 Lack of learner engagement with feedback Lack of understanding of feedback Transmitted feedback creates dependency onteacher Not utilising self/peer feedback Lack of a shared context for assessment forteacher and learner High teacher effort low efficiency Reduced staff satisfaction as evidence offeedforward not seen 7. Shift conceptions of feedback Feedback should develop the studentscapacity to make evaluative judgements abouttheir own and others work1,2 Feedback should serve the function ofprogressively enabling students to bettermonitor, evaluate and regulate their ownlearning, independently of the teacher3 8. So what did we dohttp://youtu.be/S5bBFEbXDD0 9. Re-engineered our assessment Modular approach tosequencing of assessment Explicating feedback timesin a student-tutor charter Development ofassessment rubrics for allsummative Increased use of formativeassessments 10. PG Certificate in Medical Education Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 chapters chapters chapters chapters chapters chapters chapters chapters chapters chaptersEquivalent 2Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2chapterschapters chapters chapters chapters chapters chapters chapters chapters chapters Revised PG Certificate in Medical Education Formative assessmentChapters246 810 Medium stakes assessment High stakes assessmentChapters 2 46 8 10Chapters 2 46 810Mark Russell 2010Chapters246 810 University of HertfordshireESCAPE project 11. Faculty development Workshops about feedback and importantly agreement on how to give feedback (process) 12. Include self-review Self-review is a scenario where students make evaluative judgements about the own work and produce a written feedback commentary4 13. Content (understanding of theory / principles andSelf-evaluation:application to own context)- Understanding of learning theory- Evidence of critical reflection on learning theory andkey learning and teaching principles- Evidence of application of learning theory to ownpractice Tutor feedback:Style, format and language (e.g. structure, coherence, Self-evaluation:flow, formatting, use of language) Tutor feedback:Sources and references (e.g. range of references cited,Self-evaluation:relevance, consistency, accuracy and completeness ofreferencing) Tutor feedback:Which aspect(s) of your assignment would you Student comment:specifically like feedback on? Tutor feedback:How did previous feedback inform this assignment?Student comment:Name of Tutor: Date: 14. Engaging students in processing of feedback through the wiki When you receive your assignment with feedback please upload a copy to your wiki and include a reflection on the following four questions: 1. How well does the tutor feedback match with your self-evaluation? 2. What did you learn from the feedback process? 3. What actions, if any, will you take in response to the feedback process? 4. What if anything is unclear about the tutor feedback? 15. Student engagement interACT launched 30th April 2012 100% completion of the cover page Wiki engagement varies from 65-20% 16. Evaluation interviews to date Conducted interviews with 5 students They find the instructions to be clear (questionsto technical team have dropped) They value the opportunity to ask for specificfeedback on aspects of their work and ability todialogue with tutors The cover page self-evaluation is repetitive forreferences and style sections Building on this with a survey to be launched inthe next week 17. Aspects I would like feedback on: Ideas of not preparing too rigidly in order to be flexiblewithin sessions practical advice would be welcomed! As the first essay I have written in nearly 20 years, I wouldlike to know whether the standard overall was acceptable Please advise me how I can enter a specific page number ina reference when using Endnote (I wanted to add p. 58 tothe first reference used in the text since this is an exactquote, but failed to find out how I can do it, despite usingthe help option of the software). Feedback on whether my peers have had similar thoughtsfor their own teaching, or other ideas that have beencommonly developed would be beneficial in case I have notthought or considered them. any part of it 18. How did previous feedback inform thisassignment It made me realise that instead of focusing on asingle or a few key teaching principles, I focusedon many of them without going into much detail.Also I had used bullet points in the text. Feedback that my writing style was agreeablewas reassuring. I appreciated knowing my use ofliterature was valid and supportive in the previousassignment, so have tried to continue applyingthe literature to my work. I tried to be careful to define and reference jargon It was really helpful in writing present assignment 19. I really like the cover sheet and request for selfassessment and what you would like feedbackabout ... consistent with principles of feedback!!(M&L end of module feedback)I like this better, as there is an element ofexpectation that there will be some dialogue What I really like about it, is that it gives me achance to request, to maybe direct a bit thespecificity of what kind of information I am going toget (student interview #3) 20. Added challenges of: For sharing feedback Relationship and trust Asynchronous dialogue andshared understanding Reliance on written feedback Technology 21. Benefits of doing the audit Raises awareness of quantity and quality offeedback provided individually and across thecentre Opportunity for faculty development Objective measure of changes in feedbackculture The process will be repeated on 2012 assignmentsat the beginning of 2013 22. Future developments Develop a toolkit of e-activities designed toimprove self, peer and tutor dialogic feedbackthrough the use of available technologies suchas blogs, Friendfeed, Skype/webinars, orsynchronous chat Introduce a patchwork assignment at the endof the Certificate andDiploma Work to refine process 23. Project websitehttp://blog.dundee.ac.uk/interact 24. Contact DetailsRola AjjawiCentre for Medical EducationUniversity of DundeeEmail: r.ajjawi@dundee.ac.ukTwitter: @r_ajjawihttp://blog.dundee.ac.uk/interact/Karen BartonCentre for Medical EducationUniversity of DundeeEmail: interact@dundee.ac.uk 25. References1. Sadler, D. R. (2010) Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35:5, 535-550.2. Boud, D., & Associates. (2010). Assessment 2010: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council3. Nicol, D. J., & MacfarlaneDick, D. (2006). Formative assessment and selfregulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.4. Nicol, D. (2012). Assessment and feedback - in the hands of the student [Online]. JISC. Available: http://jiscdesignstudio.pbworks.com/w/page/50118521/Assessment%2 0and%20feedback%20- %20in%20the%20hands%20of%20the%20student [Accessed 01/02/12].5. Brown, E. & Glover, C. (2006) Evaluating written feedback. in: B. C. & K. Klegg (Eds) Innovative assessment in higher education. London, Routledge), 81-91.6. Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of Educational Research, 77, 81-112.