online approaches to marking and feedback summer 2012

Online approaches to marking and feedbackBenefits and considerations for staff and students

Roger Gardner, Learning Technologist

Education Support Unit

Session outline

• Context • Purposes of assessment• Effective feedback practice

• How can technology help?• Examples• Questions and discussion

Purposes of assessment

• Assessment of learning• Assessment for learning

Sadler’s 3 conditions

3 conditions for effective feedback: “The learner has to:

1. possess a concept of the standard (or goal, or reference level) being aimed for

2. compare the actual (or current) level of performance with the standard

3. engage in appropriate action which leads to some closure of the gap”

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. http://datause.cse.ucla.edu/DOCS/drs_for_1989.pdf

Good feedback practice 1. helps clarify what good performance is

2. facilitates the development of self-assessment (reflection) in learning

3. delivers high quality information to students about their learning

4. encourages teacher & peer dialogue around learning

5. encourages positive motivational beliefs and self-esteem

6. provides opportunities to close the gap between current & desired performance

7. provides information to teachers that can be used to help shape teaching

Nicol, D & Macfarlane-Dick , D (2006) "Formative assessment and self-regulated learning: A model and seven principles of good feedback practice" Studies in Higher Education vol.31 no.2 pp.199-218

NUS 10 feedback principles

1. For learning, not just of learning

2. Continuous process

3. Timely

4. Related to clear criteria

5. Constructive

6. Legible and clear

7. Provided on exams

8. Include self and peer feedback

9. Accessible to all students

10. Flexible and suited to students’ needs

Education Support Unit

National Union of Students (2008) The great NUS feedback amnesty briefing paper

How can technology help?• Legibility of text• Varied formats e.g. audio, video• Easy access from anywhere• Students have control over when, where

and how they receive and engage with marks & feedback e.g.  in privacy at home

• Consistency e.g. online templates aligned with assessment criteria

Education Support Unit

How can technology help?• Storing feedback with other learning

resources makes it convenient for students to revisit it when completing future assignments

• Students and staff can monitor progression• Opportunities for reflection• Opportunities for self & peer assessment

Technology, Feedback, Action!: Impact of Learning Technology on Students' Engagement with Feedback , Sheffield Hallam

Types of feedback include

• Lecturer/other staff – individual student• Lecturer/other staff – group of students• Student(s) – student(s)• Student – self

Examples1. Blackboard Grade Centre for marks &

feedback

2. Feedback formats include:

a. comments in Word documents, typed and by e-pens

b. audio feedback

c. screencasts

d. rubrics

Education Support Unit

Examples

3. Onscreen marking e.g. Turnitin Grademark

4. Feedback in lectures using e-voting

5. Peer feedback e.g. Blogs / file exchange

6. Personal journals for reflection

Please choose an example ...

1. Blackboard Grade Centre

2. Word annotation tools

3. E-pens

4. Audio / video feedback

5. Rubrics

6. Onscreen marking

7. Peer feedback

8. Reflective journals

9. E-voting

Blackboard Grade Centre

Benefits include:• Central location for storage of marks and

feedback• Handles variety of types of feedback e.g.

text, audio, video• Audit trail and tracking• Feedback located with associated learning

resources

Considerations

• Different options for marks and feedback so clear workflow advisable

• No true anonymous marking (only trust-based)• Grade Centre for each BB course (unit or

programme) , but quicker course-to-course navigation coming this summer

• Batch download possible from BB but currently no batch upload of feedback

Student submission

Click on image for video

Full Grade Centre

Needs marking

Adding marks and feedback

Click on image for video

Marks and feedback in GC

Student view – My Grades

Have you used comments in Word?

1. Yes

2. No

Comments in Word

Click on image for video

Benefits• legibility and clarity• range of annotation tools available in Word

– can be used with e-pens or tablets• many people familiar with comments/track

changes in Word• ability to go back and quickly edit feedback• can mark offline and then upload e.g. to

Blackboard

Considerations

• health and safety – need to adapt marking habits , taking regular breaks

• student submissions need to be in the correct file format

• importance of feedback file naming and organisation

Handwritten comments - epens

Click on image for video

Benefits

• Not limited to typed feedback e.g. can draw , write equations etc

• e-pens plug and play in Windows 7• Work with built in Windows handwriting

recognition• Can use Ink comment function in Word /

ink handwriting in OneNote

Considerations

• For some users – getting used to the feel of the e-pen

• Training Windows to recognize your handwriting can take time

• Compare with other e-handwriting possibilities e.g. tablets

Audio feedback

Record e.g. using

Windows sound

recorder, Audacity or

digital voice recorder

Upload as feedback

to Blackboard for

student to download

Click on image for video

Reported benefits include:

• richer, more detailed feedback• seen as more personal• increased student satisfaction• increased student engagement

“Sounds good" projecthttp://sites.google.com/site/soundsgooduk/Home

Education Support Unit

Considerations

• May take practice before it saves time• Don’t spend time re-recording / editing

unless absolutely necessary• Pre-emptive recordings can be used to

address likely errors/issues/questions• One to many audio feedback could save

time and engage students

Rotheram, B. Practice tips on using digital audio for assessment feedback. Available from Sounds Good project website

ScreencastingBenefits• Many applications

e.g. worked examples, demos

• audio & video• individual or group

feedback• free easy to use

tools e.g. Screenr

• e.g. 2 http://goo.gl/DkcYn

Click on image for video

Considerations

• Free tools normally have time limit e.g. 5 mins (can be a benefit)

• Quickest workflow is to do a single take and not edit, so feedback may be “unpolished”

• Best to keep feedback short as video file sizes can get larger

Rubrics (marking criteria)• Click on links below for short

videosAdding a rubric to an assignment in Blackboard

• Student views rubric before submitting

• Marker uses rubric for marking• Student views marks & feedba

ck in rubric• Student viewing rubric when s

ubmitting to Turnitin

Rubrics

Benefits• Consistency• Marks and feedback

related to clear criteria

• Can adapt existing marking schemes and criteria

• Can be re-used

Considerations

• Need to check specific functions of different tools e.g. Blackboard rubric better suited then Turnitin to handle mark bands

Onscreen marking: Grademark

Click on image for video

Grademark

Benefits

• Drag & drop comments

• Re-use comments• Rubrics• Integrated with text-

matching

Considerations

• Not available offline• Limited licences

currently• Health and safety –

may need to adapt marking habits

Feedback in lectures using e-voting

Benefits include:• Instant feedback• Active engagement• Can be anonymous• Students can

compare with peers• Results can be

savedhttp://www.bris.ac.uk/esu/e-learning/support/tools/e-voting/

Click on image for case study

Considerations• Question design• Practical management of handsets• Availability of handsets• Be prepared to be flexible in a session (e.g.

to address misconceptions)• Allow time for discussion to explore

responses

http://cft.vanderbilt.edu/teaching-guides/technology/clickers/#challenges

Peer feedback

• File exchange example

• Word docs reviewed and shared

• Other options available in BB and elsewhere

Or video at: http://screenr.com/FcN8

Click on image for case study

Benefits• Encourage engagement with marking criteria• Promote deep learning e.g. evaluation• More efficient & timely feedback for large

groups• Students practice and transferable skills e.g.

providing constructive criticism• Helps students learn from each other and

place their own work

Considerations• Students often say they don’t like it! So

the rationale and benefits for students need to be clearly communicated

• Value of process as well as product• Moderation• Student involvement in formulating

assessment criteria• Careful selection of appropriate tool

Blogs or journals

Benefits• Opportunities for self-assessment• Student is central and active • Simple format, but can be used for:

Individual reflection (e-portfolio) Keeping notes to inform later work Keeping diary (recording data for later analysis) Feedback through comments Peer review / assessment

Considerations

• What will be participants’ motivation to blog? e.g. requirements of course / assessment /

personal value?• Audience - who are they writing for ?

public, private, course, group• Community - who will read /comment?

peers, tutor, other?

Questions?

• Contacts and further information: www.bris.ac.uk/esu

• Handout containing references:https://www.ole.bris.ac.uk/bbcswebdav/xid-2485285_4