www.phil-race.co.uk guest speaker series 2010-2011 increasing students’ satisfaction with their...
TRANSCRIPT
www.phil-race.co.uk
GUEST SPEAKER SERIES 2010-2011
Increasing students’ satisfaction with their learning experience: addressing issues raised by the
National Student Survey
Phil RaceBSc PhD PGCE FCIPD SFHEA NTF
Emeritus Professor
Leeds Metropolitan University
Name labels…Name labels…Please write your first name, big and
bold, on a coded sticky label, and stick it to your clothing (not onto a fabric which would be damaged).
A3
Phil
www.phil-race.co.uk
Every student comes with a free
person
I’m a person, not a
number
Getting to know studentsGetting to know students Asking students to write their names on
labels helps them to get to know each other, and helps us to know what they want to be called.
When they feel they’re known to us as individuals (in a good way), they’re less likely to drop out – especially in big groups in HE.
Labels are cheap and environmentally-friendly.
The codes can be used to get students into different group configurations, or to quiz the whole class.
www.phil-race.co.uk
This slide has three main This slide has three main purposes….purposes….
To focus the data projector, To focus the data projector, when necessary, and…when necessary, and…
Annoy all the folk near the Annoy all the folk near the back, and...back, and...
...to persuade colleagues ...to persuade colleagues never to put this sort of detail never to put this sort of detail onto a slide!onto a slide!
About Phil… Born a Geordie First a musician Then a writer Then a scientist Then a researcher Then a lecturer and
warden Got interested in how
students learn And the effects
assessment and feedback have on them
And how we teach them Gradually became an
educational developer
And now ‘retired’! (1995, 2009)
Working with students on learning techniques
And lecturers on teaching and assessment strategies
And trainers on training design
Currently… Emeritus Prof: Leeds Met Travelling around as usual! Based at Newcastle and
Leeds And on trains And still a Geordie.
And an expert…on train routes and timetables!
www.phil-race.co.uk
What’s an ‘Emeritus Professor?’
Someone really old? Someone who has not been sacked for gross
moral turpitude? Someone who’s still on the books, but not on
the payroll? Or perhaps as in ‘edentate’, the ‘e’ signifies
‘without’.
www.phil-race.co.uk
Welcome to this session There is a terrific range of experience of
higher education in this room – let’s bring our thinking together to find creative solutions to the problems we face in trying to achieve ‘student satisfaction’.
www.phil-race.co.uk
Thanks to students I could not do what I do nowadays if I
did not continue to spend a fair bit of my time working with students – they’ve taught me most of what I know.
Represented today... The Centre for Medical Education, Nursing and Midwifery (7 registered), Management (3), Centre for Vision and Vascular Science, Architecture, SPACE (School Office), English, Pharmacy, Biological Sciences (IAFLU), Mechanical & Aerospace Engineering, Graduate Studies, Music and Sonic Arts (2 from School
Office), University of Ulster Psychology (2), Sociology, Social Policy and Social Work
History Computer Science Geography Graduate Studies Union Theological College Marketing, Recruitment &
Admissions Educational and Skills Development AISHE reps (open to AISHE members
as event included in AISHE regional seminar series) from Dublin City University, National College of Art & Design and Dundalk Institute of Technology.
www.phil-race.co.uk
Not (known, but not shared)
www.phil-race.co.uk
You will be able to download my main slides
You don’t need to take notes.But of course do feel free to make notes of
anything you think of – your own thoughts, questions, issues and so on.
I’ll put the main slides I use today up on my website before the day is out.
So sometimes I’ll go fast.
www.phil-race.co.uk
Context In the UK, feedback is gathered nationally from
final year students by the National Student Survey (NSS), introduced in 2005, and implemented between January and March each year. There has also been discussion of the development of such a survey in the Republic of Ireland.
The NSS looks at teaching, assessment and feedback, academic support, organisation and management of teaching, learning resources and students’ personal development.
www.phil-race.co.uk
Today ... In this workshop, we address head-on
together how we can go about increasing student satisfaction in the context of the Survey, in particular in the area of assessment and feedback which students across the sector have consistently rated as the least satisfactory aspect of their experience in higher education.
www.phil-race.co.uk
Questions to be addressed1. What can I do in my teaching to increase
my students’ satisfaction with their learning?
2. How can I ensure that students’ increased satisfaction will show from their evaluation responses?
Colleagues across the sector in Ireland, both south and north, may gain some useful ideas which will help not only to improve their students’ experience, but also to improve how students express their views about that experience.
Intended learning outcomes
By the end of this interactive session, you should be better-able to
1. Identify, and address some of the principal causes of student dissatisfaction.
2. Work systematically towards improving the quality of the student experience of higher education.
3. Learn from ‘what the gurus tell us’ about assessment and feedback, and respond to this particularly important aspect of student satisfaction.
www.phil-race.co.uk
Post-it introductory task On a post-it, in your best handwriting,
please write your own short completion of:
“my students would be much more satisfied with their experience of higher education if only I …”
Please swap post-its until you don’t know whose you have.
If asked, read out with passion and drama what’s on the post-it you now have.
April 19, 2023April 19, 2023 1818
Using post-its to find out Using post-its to find out students’ real intended learning students’ real intended learning
outcomesoutcomes
Post-its…Post-its… A small, equal opportunities,
non-threatening space. Just about everyone is willing to jot
something down on a post-it in answer to a question, whereas they may not offer a spoken answer to a question, or write responses on a blank sheet of paper.
Post-its allow everyone the same opportunity to respond, including the quiet or shy students.
Post-its can be swapped, and students can read out someone else’s ideas, in the relative comfort of anonymity.
Finding out where a Finding out where a group is starting from…group is starting from…
Post-its are particularly useful for open-ended questions, such as ‘economics would be much better for me if only I …’
Responses can be posted on a flipchart or wall, and used as an exhibit.
They can be photocopied and returned to students.
Post-its can be a fast way of finding out what the real intended learning outcomes are for a group.
They can also provide a measure of the learning incomesincomes of the group.
Wednesday 19 April 2023Wednesday 19 April 2023 2121
Face-to-face communicationFace-to-face communication
Making all the channels workMaking all the channels work
Wednesday 19 April 2023 Teaching smarter (Phil Race) 22
Face-to-face one-to-one feedback activity
Please work in pairs, moving around the room, talking to different people using the script which follows…
The script:
A ‘Hello’.
B ‘Hello’.
A ‘You are late’.
B ‘I know’.
Try to do it completely differently each time.
Wednesday 19 April 2023 Teaching smarter (Phil Race) 23
The power of face-to-face communication
When explaining assessment criteria to students, and when linking these to evidence of achievement of the intended learning outcomes, we need to make the most of face-to-face whole group contexts and,,, Tone of voice Body language Facial expression Eye contact The chance to repeat things The chance to respond to puzzled looks
Some things can’t work nearly so well just on paper or on screens.
www.phil-race.co.uk
Finding out how we’re doing – seeking and using feedback
from studentsPhil Race
Visiting Professor: Assessment, Learning and Teaching, Leeds Metropolitan University
Wednesday 19 April 2023
www.phil-race.co.uk
www.phil-race.co.uk
Rationale Questionnaires are perhaps the most common way of
finding out what students think about our teaching – but probably the least effective way, as we only get responses to the questions we happen to include.
There’s no facial expression, tone of voice, eye contact nor all the rest of normal human communication.
It’s worth continuing to think of other ways of getting and using feedback from students, and share our collective experience regarding what students have told us
www.phil-race.co.uk
Some limitations of questionnaires…
Ticky-box syndrome Performing dogs syndrome Lost learning opportunities WYSIWYG syndrome Blue, rosy and purple questionnaires Conditioned response questionnaires Death by questionnaire
www.phil-race.co.uk
Feedback and assessment We’ve moved from... Assessment of learning, To assessment for learning, and Towards assessment as learning. Perhaps we need to be thinking of how
we can get feedback from students with the feedback as learning for them?
www.phil-race.co.uk
Student feedback – some questions
How can we develop student ownership?
Are we getting the right feedback? Can we make giving feedback a learning
experience in itself? How will we give students the results
(and benefits) from their feedback?
www.phil-race.co.uk
Nine ways of gathering feedback...
1. Interviews with individual students.
2. Feedback activities with groups of students.
3. Solicited feedback from large groups, e.g. ‘stop, start, continue’.
4. Questionnaires.
5. Student representation.6. Informally, through tutorials, seminars, etc.7. Students’ summative performance.8. Students’ coursework performance.9. External observers, moderators, examiners,
peer-observation, etc.
www.phil-race.co.uk
Feedback and evaluation My personal view: evaluation takes five
years – and after five years of the National Student Survey we can begin to evaluate what it’s telling us.
All we can get from our students is feedback on their experiences (I never use ‘evaluation’ forms).
We can, however, use the feedback as part of our ongoing evaluation of the student experience.
www.phil-race.co.uk
What I’m wondering... Whether it might help to get feedback
relating to the seven factors underpinning successful student learning...
www.phil-race.co.uk
Ripples on a pond….
Wanting/Needing
Doing
Feedback
Teaching
Assessing
Making sense
www.phil-race.co.uk
e.g. What could we learn from...
1. Feedback from students on their ‘want’ to learn, and how we enhance it?
2. Feedback from students on their degree of ownership of the need to learn, as defined by our intended learning outcomes?
3. Feedback from students on what we get them to do – practice, trial and error, repetition, and so on?
4. Feedback from students on how well they ‘make sense’ of what they’re learning, and what we do to help them get their heads round things?Doing
www.phil-race.co.uk
What more can we learn from students about
5. How well our feedback to them helps them – what works best, and what doesn’t work?
6. How much they learn from coaching each other, explaining things to each other, and teaching each other?
7. How much they deepen their learning by making informed judgements about their own learning, and each others’ learning?
Useful sources Knight, P and Yorke, M (2003) Assessment, learning and
employability Maidenhead, UK SRHE/Open University Press. Bowl, M (2003) Non-traditional entrants to higher education
‘they talk about people like me’ Stoke on Trent, UK, Trentham Books.
Gibbs, G (2010) Using assessment to support student learning Leeds: Leeds Met Press.
Boud, D and Associates (2010) Assessment 2020: seven propositions for assessment reform in higher education Sydney: Australian Learning and Teaching Council.
Joughin, G (2010) A short guide to oral assessment Leeds: Leeds Met Press.
Gordon Stobart (2008) Testing Times – uses and abuses of assessment London: Routledge.
My stuff Race P (2010) Making learning happen: 2nd edition London:
Sage Publications: this edition has frequent links to the statements used in the UK National Student Survey, including what we can do to increase satisfaction.
Race, P and Pickford, R (2007) Making teaching Work London: Sage Publications: this is the source material from which the handout has been adapted.
Race, P (2007) How to get a good degree: 2nd edition Maidenhead: Open University Press: this may help you to see how students think about their experience of higher education.
www.phil-race.co.uk
Individual task What bugs students? Jot down on separate post-its, three
things which you consider causes student dissatisfaction.
In clusters as you’re sitting, ‘diamond-nine’ these things, working out what bugs students most, 2nd-most, 3rd-most and so on.
Diamond-9 of things that bug Diamond-9 of things that bug students…students…
1
2 3
5 64
87
9
worst
Less bad
www.phil-race.co.uk
Your top dissatisfaction items...
Quality of teaching – concepts not explained...
Poor preparation of teaching, poor commitment.
Slow, or no feedback Lack of communication/connection
between lecturer and student Internal information systems! Not treating students with respect...
www.phil-race.co.uk
More... Over assessment Too many assessments at one time Untimeliness of feedback Irrelevant modules Poor quality learning materials Lack of availability of materials Poor organisation Students feeling that they have to work too
hard
www.phil-race.co.uk
Albert Einstein
“It is simply madness to keep doing the same thing, and expect different results”
www.phil-race.co.uk
The UK National Student Survey 2005-10
22 statements: several of which are poorly crafted. The design of this particular survey if far from ideal, but its
use right across the higher education sector in the UK makes it an important indicator of at least some of students’ feelings about their higher education experience. Students are asked to make judgments as follows:
definitely agree mostly agree neither agree nor disagree mostly disagree definitely disagree Not applicable
www.phil-race.co.uk
The main areas The teaching on my course Assessment and feedback Academic support Organisation and management Learning resources Personal development
And finally... Catch 22.Overall, I am satisfied with the quality of
this course.
www.phil-race.co.uk
The teaching on my course1. Staff are good at explaining things.
2. Staff have made the subject interesting.
3. Staff are enthusiastic about what they are teaching.
4. The course is intellectually stimulating.
www.phil-race.co.uk
Assessment and feedback5. The criteria used in marking have been
clear in advance.
6. Assessment arrangements and marking have been fair.
7. Feedback on my work has been prompt.
8. I have received detailed comments on my work.
9. Feedback on my work has helped me clarify things I did not understand.
www.phil-race.co.uk
Academic support10. I have received sufficient advice and
support with my studies.
11. I have been able to contact staff when I needed to.
12. Good advice was available when I needed to make study choices.
www.phil-race.co.uk
Organisation and management
13. The timetable works efficiently as far as my activities are concerned.
14. Any changes in the course or teaching have been communicated effectively.
15. The course is well organised and is running smoothly.
www.phil-race.co.uk
Learning resources16. The library resources and services are
good enough for my needs.
17. I have been able to access general IT resources when I needed to.
18. I have been able to access specialised equipment, facilities, or rooms when I needed to.
www.phil-race.co.uk
Personal development19. The course has helped me to present
myself with confidence.
20. My communication skills have improved.
21. As a result of the course, I feel confident in tackling unfamiliar problems.
And...
22. Overall, I am satisfied with the quality of this course.
www.phil-race.co.uk
Fishing for feedback?
Feedback is like fish.
If it is not used quickly, it becomes useless.
(Sally Brown).
Give a man a fish,
Feed him for a day.
Teach a man to fish,
Feed him for a lifetime.
(Chinese proverb).
Make feedback timely, while it still matters to students, in time for them to use it towards further learning, or to receive further assistance.
(Graham Gibbs)
www.phil-race.co.uk
The ten most important words
But you have to remember this bit – I don’t give away the 10th word in case it leaks out to where I’m going next.
www.phil-race.co.uk
Context Assessment and feedback take up a great deal
of our time and energy – and also a great deal of students' time and energy.
Yet it can be argued that assessment and feedback are broken in higher education nowadays. They take too much of our time and we don’t always measure the right things.
www.phil-race.co.uk
We desperately need to make assessment work better
In the UK, we know from the National Student Survey that since 2005 the evidence suggests that students nationally find assessment among the least satisfactory elements of their experience of higher education. We also know that assessment take up ever more of our time and energy.
www.phil-race.co.uk
Making exams fun – what else can we do?
1. Read out the instructions to candidates – rather differently!
2. Include a silly question (just for one mark).
3. Give the senior invigilator a brick.
4. Ask for a volunteer candidate to accompany the invigilator to the toilet?
www.phil-race.co.uk
Assessment is broken in higher education
Student numbers have grown: we can’t use the same processes and instruments for a system where getting on for 50% of the 18-30 year old population study in post-compulsory education, compared to 5% a couple of decades ago.
The world has opened up, so that our assessment processes and practices need to be more compatible with those in quite different cultures and traditions.
www.phil-race.co.uk
And more... It is widely accepted now that assessment is
the major driver for student learning, and if assessment is not working as a good driver for learning, the effectiveness of our entire higher education provision is jeopardised.
We need to continue to diversify the assessment processes and instruments we use, so that no students are repeatedly disadvantaged by the predominance of particular assessment formats.
www.phil-race.co.uk
In short, We assess far too much, Using the same old ways far too often, Assessment takes far too much of our
time – and far too much time of our students.
And we drive down the quality of learning by our assessment.
www.phil-race.co.uk
And what’s wrong with feedback?
Students get it too late. And too often, it’s just words on paper. It doesn’t help them enough. They often don’t take enough notice of
it.
www.phil-race.co.uk
But what’s really wrong with feedback?
It’s one-way – monologic. What students want is dialogue. They want to talk to us about their work. But they’re scared to talk to us, for
various reasons:In case it leads to lower marks;In case they’re ‘found out’;In case they feel stupid.
www.phil-race.co.uk
What the gurus tell us on assessment,
feedback and learning
Towards making learning happen better, using smarter assessment
and feedback
www.phil-race.co.uk
Sally Brown, 2009
Concentrating on giving students detailed and developmental formative feedback is the single most useful thing we can do for our students, particularly those who have had a struggle to achieve entry to higher education.
Assessment and feedback are two of the best tools available to us, to support student achievement, progression and retention.
www.phil-race.co.uk
“With the changing economy, no one has lifetime employment, but community colleges provide lifetime employability”.
President Barack Obama
Gibbs, 2010: tactics to improve student learning
1. Capture student time and effort, distributing that effort appropriately across topics and weeks.
2. Generate high-quality learning effort, oriented towards clear and high standards.
3. Provide sufficient feedback, often enough, and in enough detail.
4. Focus feedback on students’ performance, on actions under their control, rather than on students themselves or their characteristics.
www.phil-race.co.uk
Gibbs, 2010: tactics to improve student learning
5. Make feedback timely, while it still matters to students, in time for them to use it towards further learning, or to receive further assistance.
6. Link feedback to what students believe they are supposed to be doing.
7. Ensure that feedback is not only received, but is attended to, so that students act on it to change their future learning and performance.
www.phil-race.co.uk
Doing
Ripples on a pond….
Wanting/Needing
Doing
Feedback
Assessing
making informed
judgements
Making sense
Coaching, explaining, teaching
1. Strive to enhance our students’ want to learn;
2. Help students to develop ownership of the need to learn;
3. Keep students learn by doing, practice, trial-and-error, repetition;
4. Ensure students get quick and useful feedback – from us and from each other;
5. Help students to make sense of what they learn.
6. Get students deepening their learning by coaching other students, explaining things to them.
7. Allow students to further deepen their learning by assessing their own learning, and assessing others’ learning – making informed judgements.
Smarter assessment and feedback: we need to use them to:
www.phil-race.co.uk
Life is too short to… Spend time and energy writing feedback which
won’t actually be used by students (sometimes not even collected by them);
Write feedback just for external examiners to see.
Approach giving feedback only in the ‘read-write’ dimension, when many students gain more from it through auditory, or visual, or kinaesthetic channels.
(see Neil Fleming’s excellent (and free!) ‘VARK’ work on www.vark-learn.com).
www.phil-race.co.uk
The read-write problem Higher education has become a read-write
industry!
Yet only since 1791 have we been using written exams and written feedback.
Feedback on paper is probably the most time-wasting, least effective and most dangerous way to give students feedback.
There is still a lot going for oral assessment and feedback – see Joughin 2010.
www.phil-race.co.uk
Boud et al 2010: ‘Assessment 2020’Assessment has most effect when...:1. It is used to engage students in learning that is
productive.
2. Feedback is used to actively improve student learning.
3. Students and teachers become responsible partners in learning and assessment.
4. Students are inducted into the assessment practices and cultures of higher education.
5. Assessment for learning is placed at the centre of subject and program design.
6. Assessment for learning is a focus for staff and institutional development.
7. Assessment provides inclusive and trustworthy representation of student achievement.
Boud et al, 2010David Boud (University of Technology, Sydney), Royce Sadler (Griffith University), Gordon Joughin (University of Wollongong), Richard James (University of Melbourne), Mark Freeman (University of Sydney), Sally Kift (Queensland University of Technology), Filip Dochy (University of Leuven), Dai Hounsell (University of Edinburgh), Margaret Price (Oxford Brookes University), Tom Angelo (La Trobe University), Angela Brew (Macquarie University), Ian Cameron (University of Queensland), Denise Chalmers (University of Western Australia), Paul Hager (University of Technology, Sydney), Kerri-Lee Harris (University of Melbourne), Claire Hughes (University of Queensland), Peter Hutchings (Australian Learning and Teaching Council), Kerri-Lee Krause (Griffith University), Duncan Nulty (Griffith University), Ron Oliver (Edith Cowan University), Jon Yorke (Curtin University), Iouri Belski (RMIT University), Ben Bradley (Charles Sturt University), Simone Buzwell (Swinburne University of Technology), Stuart Campbell (University of Western Sydney), Philip Candy (University of Southern Queensland), Peter Cherry (Central Queensland University), Rick Cummings (Murdoch University), Anne Cummins (Australian Catholic University), Elizabeth Deane (Australian National University), Marcia Devlin (Deakin University), Christine Ewan (Australian Learning and Teaching Council), Paul Gadek (James Cook University), Susan Hamilton (University of Queensland), Margaret Hicks (University of South Australia), Marnie Hughes-Warrington (Monash University), Gail Huon (University of Newcastle), Margot Kearns (University of Notre Dame, Sydney), Don Maconachie (University of the Sunshine Coast), Vi McLean (Queensland University of Technology,) Raoul Mortley (Bond University), Kylie O’Brien (Victoria University), Gary O’Donovan (University of Tasmania), Beverley Oliver (Curtin University), Simon Pyke (University of Adelaide), Heather Smigiel (Flinders University), Janet Taylor (Southern Cross University), Keith Trigwell (University of Sydney), Neil Trivett (University of Ballarat), Graham Webb (University of New England).
www.phil-race.co.uk
David Nicol et al...Good feedback practice:1. Helps clarify what good performance is (goals, criteria,
expected standards);2. Facilitates the development of self-assessment
(reflection) in learning;3. Delivers high quality information to students about
their learning;4. Encourages teacher and peer dialogue around learning;5. Encourages positive motivational beliefs and self-
esteem;6. Provides opportunities to close the gap between
current and desired performance;7. Provides information to teachers that can be used to
help shape the teaching.
Nicol, D J and Macfarlane-Dick: Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education (2006), Vol 31(2), 199-218
www.phil-race.co.uk
The guru Royce Sadler
“The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it”. (Sadler 1989), my italics)
www.phil-race.co.uk
Royce Sadler......is the most cited author on formative feedback,
writing about it since the mid-1980s. He is doing his absolute best work now.
Sadler, D R (2009) Indeterminacy in the use of preset criteria for assessment and grading Assessment and Evaluation in Higher Education 34:2 159-179
Sadler, D R (2009) Grade integrity and the representation of academic achievement Studies in Higher Education, 34:7, 807-826
Sadler, D R (2007) Perils in the meticulous specification of goals and assessment criteria Studies in Higher Education I34:7 807-26.
Sadler, D R (2005) Interpretations of criteria-based assessment and grading in higher education Assessment and Evaluation in Higher Education 30 175-194.
Sadler, D R (2002) Ah! ... So that’s ‘Quality’ In Schartz, P and Webb, G (eds) Assessment Case Studies: experience and practice from higher education London, Kogan Page.
www.phil-race.co.uk
The Weston Manor Group 2007
ASKe: Assessment Manifesto for Change
Professor Trudy Banta, Dr Simon Barrie, Professor Sally Brown, Cordelia Bryan, Dr Colin Bryson, Jude Carroll, Professor Sue Clegg, Professor Linda Drew, Professor Graham Gibbs, Dr Karen Handley, Professor Anton Havnes, Dr Mary Lea, Dr Janet Macdonald, Professor Ranald Macdonald, Dr Debra Macfarlane, Dr Susan Martin, Professor Marcia Mentkowski, Dr Stephen Merry, Professor David Nicol, Professor Andy Northedge, Professor Lin Norton, Berry O’Donovan, Dr Thomas Olsson, Dr Susan Orr, Dr Paul Orsmond, Professor Margaret Price, Professor Phil Race, Clive Robertson, Dr Mark Russell, Dr Chris Rust, Professor Gilly Salmon, Professor Kay Sambell, Professor Brenda Smith, Professor Stephen Swithenby, Professor Mantz Yorke.
www.phil-race.co.uk
The Weston Manor Group:Assessment Standards: a Manifesto for
change1. The debate on standards needs to focus on how high
standards of learning can be achieved through assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning.
2. When it comes to the assessment of learning, we need to move beyond systems focused on marks and grades towards the valid assessment of the achievement of intended programme outcomes.
www.phil-race.co.uk
3. Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment.
4. Assessment standards are socially constructed so there must be a greater emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards. It is when learners share an understanding of academic and professional standards in an atmosphere of mutual trust that learning works best.
Wednesday 19 April 2023
The Weston Manor Group:Assessment Standards: a Manifesto for
change
www.phil-race.co.uk
5. Active engagement with assessment standards needs to be an integral and seamless part of course design and the learning process in order to allow students to develop their own, internalised conceptions of standards, and to monitor and supervise their own learning.
6. Assessment is largely dependent upon professional judgement, and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities.
Wednesday 19 April 2023
The Weston Manor Group:Assessment Standards: a Manifesto for
change
www.phil-race.co.uk
Assessment Reform Group 2002
Testing, motivation and learningProfessor Paul Black King’s College, London
Professor Patricia Broadfoot University of Bristol
Professor Richard Daugherty University of Wales, Aberystwyth
Professor John Gardner Queen’s University, Belfast
Professor Wynne Harlen University of Bristol
Dr Mary James University of Cambridge
Dr Gordon Stobart Institute of Education, London
Professor Dylan Wiliam King’s College, London
Assessment Reform Group 2002Do more of this ...
Provide choice and help pupils to take responsibility for their learning. Discuss with pupils the purpose of their learning and provide feedback that will
help the learning process. Encourage pupils to judge their work by how much they have learned and by
the progress they have made. Help pupils to understand the criteria by which their learning is assessed and
to assess their own work. Develop pupils’ understanding of the goals of their work in terms of what they
are learning; provide feedback to pupils in relation to these goals. Help pupils to understand where they are in relation to learning goals and how
to make further progress. Give feedback that enables pupils to know the next steps and how to succeed
in taking them. Encourage pupils to value effort and a wide range of attainments. Encourage collaboration among pupils and a positive view of each others’
attainments.
www.phil-race.co.uk
Define the curriculum in terms of what is in the tests to the detriment of what is not tested.
Give frequent drill and practice for test taking. Teach how to answer specific test questions. Allow pupils to judge their work in terms of scores or grades. Allow test anxiety to impair some pupils’ performance (particularly
girls and lower performing pupils). Use tests and assessment to tell students where they are in relation
to others. Give feedback relating to pupils’ capabilities, implying a fixed view
of each pupil’s potential. Compare pupils’ grades and allow pupils to compare grades, giving
status on the basis of test achievement only. Emphasise competition for marks or grades among pupils.
Assessment Reform Group 2002Do less of this ...
www.phil-race.co.uk
What else can we do? In groups as directed, please think
creatively, ‘out of the box’ about how else we can act to cause our students to feel greater satisfaction about selected items on the National Student Survey.
Compose short, sharp recommendations for action, carefully, and write them on separate post-its, containing the statement number(s) addressed most directly.
www.phil-race.co.uk
Homework I’ll put the best five recommendations I
receive by 1200 on Monday 25th October as a separate file in the same post as this on my website.
www.phil-race.co.uk
Fishing for feedback?
Feedback is like fish.
If it is not used quickly, it becomes useless.
(Sally Brown).
Give a man a fish,
Feed him for a day.
Teach a man to fish,
Feed him for a lifetime.
(Chinese proverb).
Make feedback timely, while it still matters to students, in time for them to use it towards further learning, or to receive further assistance.
(Graham Gibbs)
www.phil-race.co.uk
How to get feedback to a large group of students within 24
hours There are serious reservations about
written feedback, but we can make even this work much better than it did.
The next two slides are about a way of giving students feedback on their work within 24 hours of them doing it.
There are three or more ‘yes, buts’s with this idea, but please hold these for around four minutes.
www.phil-race.co.uk
Set the hand-in time to be at a whole-group session
E.g. next Tuesday’s 10-11 lecture, Deadline = 1003. Let them pile up all their work at the
front. At 1003, issue a coloured sheet,
containing numbered points (so you can say in feedback ‘please see point 3, blue sheet’ and so on).
www.phil-race.co.uk
The blue sheet can contain....
Illustration of what is expected as evidence of achievement of each of the intended learning outcomes
Likely mistakes Features of a good answer Frequently needed explanations
Let the students study this for 3 minutes until 1006 – it goes rather quiet!
www.phil-race.co.uk
At 1006...
Go into face-to-face oral mode, until 1009....
Spend a few minutes de-briefing the whole group and talking them through one point on the handout…
…adding tone-of-voice, facial expression, body language, emphasis, and so on to the feedback.
www.phil-race.co.uk
The rationale...
Since many students will have done the work in the last 24 hours before handing it in, you’re giving them feedback while they still remember what they were doing.
They know what they didn’t do. They know what they missed out
because they couldn’t understand it.
www.phil-race.co.uk
Now take away their work to mark – in a third of the time it
used to take you! You waste far less writing the same old
things on one piece of work after another, regarding frequently occurring mistakes;
You can make your comments relate more to each individual piece of work;
This means when students get their marked work back with feedback, they are more likely to use it, as it’s personal to them.
www.phil-race.co.uk
Furthermore... When they get their marked work back,
they’ve already had the chance to make sense of their own piece of work in the light of the generic feedback you gave them when they handed it in, and the additional oral ‘deepening’ of one important point.
www.phil-race.co.uk
Back to our intended learning outcomesDo you now feel better able to...
(two hands = much better, one hand = somewhat better, no hands = no better)
1. Identify, and address some of the principal causes of student dissatisfaction?
2. Work systematically towards improving the quality of the student experience of higher education?
3. Learn from ‘what the gurus tell us’ about assessment and feedback, and respond to this particularly important aspect of student satisfaction?
Action planning statements
One thing I’m going to do is… One idea I’m taking away is… I’m going to think more
about… I have found out that … I’d like to know … In future, I’m not going to…
Wednesday 19 April 2023
