university of the arts workshop fair assessment 16 january 2012 sally brown emerita professor,...

University of the Arts workshop

Fair Assessment 16 January 2012

Sally Brown

http://sally-brown.netEmerita Professor, Leeds Metropolitan University,

Adjunct Professor, University of the Sunshine Coast, Central Queensland and James Cook University Queensland

Visiting Professor, University of Plymouth and Liverpool John Moores University & Honorary Fellow, University of

Northumbria

Fairness and good academic conduct

Students hate unfair assessment and are more inclined to cheat and plagiarise if they consider it to be happening. In this workshop, we will consider how assessment can be designed and managed fairly. By the end of the workshop, participants will have had opportunities to :

Explore key elements of what students perceive as unfairness;

Discuss how the setting and marking of assignments can be undertaken to maximise transparency and justice;

Consider how best to foster good academic conduct.

What students think...(Flint and Johnson, 2011, p.2)

Student evaluations frequently reveal poor assessment practices that:

1. Lack authenticity and relevance to real world tasks;

2. Make unreasonable demands on students;

3. Are narrow in scope;

4. Have little long-term benefit;

5. Fail to reward genuine effort;

6. Have unclear expectations and assessment criteria;

7. Fail to provide adequate feedback to students;

8. Rely heavily on factual recall rather than on higher-order thinking and problem-solving skills.

What do students perceive as unfairness? Perceived favouritism and bias; Poor inter-tutor reliability, where work of the same

quality receives different marks from different tutors and high marks depend on the ‘luck of the draw’ in terms of who marks your efforts;

Hidden criteria and lack of transparency; Lack of clarity about the rules of the game,

particularly in relation to students from disadvantaged backgrounds;

Complexities around terms such as ‘authentic’, ‘original’ and ‘innovative’.

We need to play fair with students by: Having ground rules to ensure equivalent (not

identical) treatment of students; Using joint assignment design, co-marking, effective

moderation, reference to agreed benchmarks and other means to assure inter-tutor reliability;

Making assessment criteria available, accessible and understandable;

Ensuring rules about compensation, late submission and xx are applied equally;

Allocating time within the curriculum to discuss and develop academic literacy in relation to contested terms.

We must play fair with students by avoiding using ‘final language’ (Boud)

Avoid destructive criticism of the person rather than the work being assessed.

Try not to use language that is judgmental to the point of leaving students nowhere to go.

Words like “appalling”, “disastrous” and “incompetent” give students no room to manoeuvre.

However, words like ”incomparable” and “unimprovable” don’t help outstanding students to develop ipsatively either.

And play fair by giving feedback to students with diverse abilities

Students at the top end of the ability range sometimes feel short changed by minimal feedback;

Students with many weaknesses easily become dispirited if there is too much negative feedback;

Consider giving an assessment sandwich. Start with something positive, go into the detailed critique and find something nice to say at the end (to motivate them to keep reading!);

Explore ways to incentivise reading of feedback; Consider which medium to use for students with

disabilities (e.g. don’t use bad handwriting for those with visual impairments or dyslexia!).

Cheating and plagiarism: why have concerns about them increased?

Mass access to HE. More reported bad practice. Changes in assessment practice. Wider use of communication and information

technologies, especially the web. Higher stakes: the importance of getting good

grades.

What is cheating? A deliberate process to ‘get round’ the system

Newstead et al have identified a whole range of behaviours which could be regarded as cheating including: making up references, taking unauthorised material into exams, impersonation, getting special help from a tutor, question spotting, recycling assignments submitted on other courses, getting extensions with false claims of illness etc

Newstead SE, Franklyn-Stokes A and Armistead P(1996) ‘Individual differences in student cheating’ Journal of Educational Psychology 88 p229-241

What is plagiarism? Passing off someone else’s work as your own. Wholesale ‘lifting’ of entire assignments/ texts. Patching and paraphrasing. Purchasing or commissioning work.

How do new technologies change things?

“Plagiarism used to be hard work…hours in the library, researching what to copy..the plagiarist used to learn a lot while trying to get out of doing the work”

Jim Evans, University of Warwick, quoted by Jude Carroll, Oxford Brookes University.

What about unintentional plagiarism? Readers may not be conscious how much they have

themselves absorbed; Schools may encourage the learning and re-use of

model answers; In some cultures, your teacher or text books are

honoured sources and there is nothing in appropriate about repeating their words.

Four strategies to control plagiarism and cheating Use strict controls. Make the rules clear and have known penalties (and

apply them). Design assessment instruments that make cheating

difficult. Develop a climate that will reduce the likelihood of

cheating.

Use strict controls Use well-invigilated, unseen, closed book exams; Use computer based tests in a strict environment; Conduct unannounced spot tests; Check for mark discrepancies; Check identities of students being assessed; Install strict controls for high stakes assessments

and ignore the rest.

Use a plagiarism detection service

In the UK, a JISC-supported national service is located at Northumbria university.

It uses a traffic light system to compare electronically submitted material against a very large number of electronic sources (red = very likely to be plagiarised, green unlikely to be plagiarised).

http://www.submit.ac.uk/static_jisc/ac_uk_index.html

Clear rules / known penalties

Brief students so they understand what we expect (and be honest about grey areas).

Publicise occurrences of cheating and associated disciplinary action (without scapegoating).

Use students to police each other (without encouraging a ‘sneak’ culture).

Foster good academic conduct

Keep excellent records for moderation, e.g. video live assignments in the studio;

Use open–book rather than closed book exams; Design assessment instruments that make cheating

difficult Use assignments reliant on personal experience; Ask students to produce learning/reflective journals

and critical incident accounts; Use vivas and orals. Design differentiated assignments.

More anti-plagiarism assignments Provide assignments with choice and individual

activity. Use computer based assessment. Involve 3rd party verification. Ask them to provide photocopied annotated source

material.

Further improving the climate Help students to feel they have some control over

their learning; Recognise competing pressures on student time and

help students who are struggling; Foster student motivation; Integrate assessment with learning; Use group assessment. Involve an element of peer assessment. Give students tasks in learning teams. Monitor the production of assessed work: use

staged assignments.

Conclusions No system is completely fair, but university teachers

need to demonstrate we are committed to academic integrity;

Students who are determined to breach good academic conduct will often find a way to do so, despite our best efforts, so we need to remain vigilant while designing good systems;

We need to ensure students have trust in the fairness and justice of the academic systems within which they are working.

Useful references: 1

Brown, S. Rust, C & Gibbs, G (1994) Strategies for Diversifying Assessment Oxford Centre for Staff Development.

Brown, S. and Glasner, A. (ed.) (1999) Assessment Matters in Higher Education, Choosing and Using Diverse Approaches, Maidenhead: Open University Press.

Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London: Kogan Page.

Carroll J and Ryan J (2005) Teaching International students: improving learning for all Routledge SEDA series

Falchikov, N (2004) Improving Assessment through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education, London: Routledge.

Flint, N R and Johnson Bruce (2011) Towards fairer university assessment: addressing the concerns of students Routledge.

Gibbs, G (1999) Using assessment strategically to change the way students learn, In Brown S. & Glasner, A. (eds.), Assessment Matters in Higher Education: Choosing and Using Diverse Approaches Maidenhead: SRHE/Open University Press.

Useful references 2

Hunt, L and Chalmers, D. (2012) University Teaching in Focus: a learning-centred approach, Abingdon, Routledge

Kneale, P. E. (1997) The rise of the "strategic student": how can we adapt to cope? in Armstrong, S., Thompson, G. and Brown, S. (eds) Facing up to Radical Changes in Universities and Colleges, 119-139 London: Kogan Page.

McDowell E & Brown S 1998 Assessing students: cheating and plagiarism, Red Guide 10/11 University of Northumbria, Newcastle

Price, M, Rust, C., Donovan, B., and Handley, K. with Bryant, R. (2012) Assessment Literacy: the foundation for Improving student learning, Oxford, Oxford Centre for Staff and learning Development.

Pickford, R. and Brown, S. (2006) Assessing skills and practice London: Routledge.

Race, P. (2001) A Briefing on Self, Peer & Group Assessment in LTSN Generic Centre Assessment Series No 9 LTSN York. Race P. (2006) The lecturer’s toolkit (3rd edition) London: Routledge.

Useful references 3

Race P (2006) The Lecturers toolkit 3rd edition London RoutledgeRace P and Pickford R (2007) Making Teaching work: Teaching

smarter in post-compulsory education, London, SageRust, C., Price, M. and O’Donovan, B. (2003). Improving students’

learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education. 28 (2), 147-164.

K Sambell, S Brown, L McDowell, (1997) ‘But is it fair? An exploratory study of stduent perceptions of the consequential validity of assessment.’ Studies in Educational Evaluation v23 n4 p349-71

Sambell, K., McDowell, L. and Montgomery, C. (2012) Assessment for Learning in Higher Education Abingdon, Routledge

Stefani L and Carroll J (2001)A Briefing on Plagiarism http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=full_record&section=generic&id=10

These and other slides will be available on my website at www.sally-brown.net