Assessing Students in Clinical Practice

Assessment

Who can supervise and assess?

Continuous assessment

Assessment validity

Reliability

Methods and strategies of assessment

Giving effective feedback

Failing a student

Assessment – what is it for?

To motivate students to learn

To punish those who do not!

To provide feedback – ‘it’s about getting to know students and the quality of their learning’ (Rowntree 1987)

To improve the quality of the learning

As a ‘quality control’ check for our teaching

With vocational courses, to ensure graduates are ‘fit for purpose’ as well as ‘fit for award’

Assessment – our task for today is to consider how we address the following:

Applying the principles and stages of the assessment process to the effective assessment of learners in practice.

Providing constructive feedback to facilitate the enhancement of learner performance.

Managing the assessment process in challenging situations.

Critically examining mentor accountability in relation to assessing learners.

Assessment of practical skills – why?

Practical skills are central to professional practice

It defines what students take to be important (Rowntree 1987)

If delegated to staff in placements:

Lack of consistency between assessors

Seen as less important than other subjects

To assess competence

Competence

The acquisition of knowledge, skills and abilities at a level of expertise sufficient to be able to perform in an appropriate work setting (Harvey 2004)

Competence - what the person is capable of doing

Performance - what the person does in his or her day-to-day practice

One needs to be competent in order to assess competence; professionals need to be assessed by professionals.

Conscious competence model

Unconscious competence

Conscious competence

Conscious incompetence

Unconscious incompetence

Benner (1984)

Expert

Proficient

Competent

Advanced beginner

Novice

Standard to Support Learning and Assessment in Practice

Agreed Mandatory Requirements

First published August 2006.

2nd edition published July 2008

Developmental framework

Specific outcomes for:

Mentors

Practice teachers

Teachers

Remaining the same

All Registered Nurses are required to supervise & facilitate the needs of learners

(NMC Code of Conduct, 2008)

NMC Code

“This Code of Conduct should be considered with the Nursing and Midwifery Council’s rules, standards, guidance and advice.”

“You must facilitate students and others to develop their competence.’’

“Failure to comply with this Code of Conduct may bring your fitness to practise into question and endanger your registration.”

(NMC 2008b)

8 Standard domains

1) Establishing effective working relationships

2) Facilitation of Learning

3) Assessment & Accountability

4) Evaluation of Learning

5) Creating an environment for learning

6) Context of Practice

7) Evidence Based Practice

8) Leadership

The Developmental Framework

The NMC standards to support learning and assessment in practice

Stage 1 Registrant

Stage 2 Mentor

Stage 3 Practice Teacher

Stage 4 Teacher

8765

4321

Five

Principals

A B C D E

Mentor Standard

2.1 “An NMC mentor is a registrant who, following successful completion of an NMC approved mentor preparation programme-or comparable preparation that has been accredited by an AEI as meeting the NMC mentor requirements-has achieved the knowledge, skills and competence required to meet the defined outcomes” (P.17).

Scenario

You are working with Amanda, a second year student nurse who commenced in your area five days ago. Amanda’s mentor has been off sick since her commencement on placement. A cardiac arrest occurs in the area and you instruct Amanda to call the cardiac arrest team and to collect the emergency equipment to commence resuscitation. She becomes very flustered and says “I don’t know what to do!”

Scenario

Focus Group Activities:

1)After the incident – describe how you would manage Amanda’s distress.

2)Consider and identify why this situation may have occurred.

3)How will you address the lack of mentor support for this student?

4)Develop a plan of action which would minimise the risk of this incident recurring.

Ideas you might have considered in relation to the student...

Commentary:

There may be a requirement to review existing or develop new orientation and induction processes for students coming to your area.

Guidelines may be a useful source of information in determining what critical topics need to be addressed and when in any induction/orientation.

Continued…

Ideas you might have considered in relation to the student...

You may also need to consider the efficacy of the initial assessment of a student’s knowledge and skills, particularly in relation to their previous experience in this environment, including fears as well as their stage of learning/development.

It may also be useful to consider mechanisms which would ensure students continue to be appropriately supported at all times during their placement.

Ideas you might have considered in relation to yourself…

Plan your future learning needs based on this scenario.

Think about and share your reactions on this scenario with your mentor other colleagues.

Reflect on and reinforce your learning from this learning experience.

Reflect on other learning outcomes which could relate to this experience.

Applying the standard in practice

Allocated learning time for mentor activity

“All students must be supervised at all times, either directly or indirectly, by a mentor/practice teacher” (3.2.3 p.29).

40% of a student’s time, when providing direct care, should be spent being supervised by a mentor or practice teacher.

Mentors are accountable for decisions relating to the delegation of activities to students.

Assessing learning in practice

“Mentors will have been prepared to assess students in practice and will be accountable for their decisions to pass refer or fail a student” (p.31).

Use evidence from various sources for making a judgment on performance.

Seek advice and guidance from others when making complex judgments.

5 required attributes of an assessment process (McKinley et al. 2001)

Reliability – consistency of assessors rating the same performance

Validity – degree to which the assessment assesses what should be assessed

Face validity often high but are we assessing what we should or what we can assess?

Acceptability – to all stakeholders (assessors, student and the public)

Continued…

5 required attributes of an assessment process (McKinley et al. 2001)

Feasibility – can it be delivered to all who need to be assessed within the cost constraints (time & staff)

Educational impact – the degree to which the assessment will help the student to improve his or her performance. This requires:

Feedback on strengths & weaknesses

Strategies for improvement

Reliability

Simulation

Agreed checklists mean less subjectivity

Criteria for assessment clearly defined

Moderator to ensure fairness and consistency

Can be video recorded

The student feels watched

Usually one-off performance and may be a ‘bad day’

Continued…

Reliability

Workplace

Wide variety of assessors involved

Student’s previous performance may influence the assessment

Have to ‘do it our way’

Informal assessment usually occurs on several occasions before it is formalised

The student may not realise he or she is being assessed

Practitioners sometimes ‘fail to fail’

Validity – Simulation

Simulated setting which may not feel real, despite good simulation.

Advantages those who can act.

Students do not know their patient; may know assessor.

Able to assess skills that are not available ‘to order’ in the workplace e.g. emergency resuscitation.

Continued…

Validity – Simulation

Environment can be controlled to the level of the student

Assessors trained for and observed during assessment

Fair – all students do the same assessment

Validity – workplace

Real workplace - authentic assessment.

Students know their patients/clients & the assessor.

Safety takes precedence – cannot allow student to make mistakes.

Reliant on the experience/patients available at the time.

Assessment may vary considerably between students

The competence of the assessors is assumed

Experts in practice does not necessarily make an expert assessor

Acceptability - workplace

Often viewed as easier than assessment using simulation

Not a ‘one-off’ performance

Includes an element of self assessment – can usually choose when to be assessed

Receive immediate feedback on their performance

Types of assessment

Formative assessment vs Summative assessment

Continuous assessment

Formal and Informal assessment

Portfolio – not truly an assessment, merely a collection of evidence, reflections etc. Why should students keep a portfolio?

Self and Peer assessment

Testimony or Witness statements

Norm referencing vs criterion referencing

"Best practice in grading in higher education involves striking a balance between criterion-referencing and norm-referencing “ James et al (2002)

Norm referencing - The main purpose behind the use of norm-referenced assessment is to rank the performance of students in a particular group in order to generate a final grade.

Criterion referencing method involves "determining a student's grade by comparing his or her achievements with clearly stated criteria for learning outcomes and clearly stated standards for particular levels of performance" (James et al, 2002).

Determining Learning Needs

How can you prepare for a student about to start a placement?

What are the main issues you should cover at the initial interview?

How do you prepare for the Intermediate interview

How can you prepare for the final interview?

Suggest ways by which you and your colleagues may attempt to reduce inconsistencies amongst assessors, thereby increase the validity and reliability of your assessment?

Common problems experienced by students

Difficulty learning during clinical practice.

Managing the situation when a student has to be failed.

References

Allin L and Turnock C (2007a) Assessing Student Performance in Work-Based Learning. Making Practice-Based Learning Work. Available: : www.practicebasedlearning.org

Allin L and Turnock C (2007b) Reflection On and In the Workplace. Making Practice-Based Learning Work. Available: : www.practicebasedlearning.org

Allin L and Turnock C (2007c) Working with others in the workplace. Making Practice Based Learning Work. Available www.practicebasedlearning.org/resources/materials/intro.htm

References

Bray L and Nettleton (2007) Assessor or mentor? Role confusion in professional education. Nurse Education Today Nov, 27(8): 848- 855.

Calman L, Watson R, Norman I, Redfern S and Murrells T (2002) Assessing practice of student nurses: methods, preparation of assessors and student views. Journal of Advanced Nursing 38(5), pp 516-523.

References

Cassidy S (2009) Subjectivity and the valid assessment of pre-registration student nurse clinical learning outcomes: Implications for mentors. Nurse Education Today 29(1), pp 33 – 39

Clarke A, Gibb A, Ramprogus A(2003) Clinical learning environments: an evaluation of an innovative role to support pre-registration nursing placements. Learning in Health and Social Care 2(2), pp 105 – 115

References

Cowan D, Norman I, Coopamah V (2005) Competence in nursing practice: a controversial concept: a focused review of literature. Nurse Education Today 25(5), pp 355 – 362

Daly W and Carnwell R. (2001) The case for a multi-method approach. Nurse Researcher 8 (3), 30–44

Duffy K (2003) Failing students: a qualitative study of factors that influence the decisions regarding assessment of students’ competence in practice. London: NMC

References

Gopee N (2008) Assessing student nurses’ clinical skills: the ethical competence of mentors. International Journal of Therapy and Rehabilitation 15(9), pp 401 – 407

Hutchings A, Williamson GR, Humphries A (2005). Supporting learners in practice: capacity issues. Journal of Clinical Nursing 14, pp 945 – 955

James, R., McInnes, C. & Devlin, M. (2002). Assessing learning in Australian universities. Canberra: Centre for the Study of Higher Education, Melbourne University and the Australian Universities Teaching Committee.

References

March S, Cooper K, Jordan G, Merrett S, Scammell J, Clark V (no date) Assessment of Students in Health and Social Care: Managing Failing Students in Practice. Making Practice-Based Learning Work. Available: www.practicebasedlearning.org

McKinley R, Fraser R and Baker R (2001) Model for directly assessing and improving clinical competence and performance in revalidation of clinicians British Medical Journal 322:712-715

References

Neary M. (2001) Responsive assessment: assessing student nurses' clinical competence. Nurse Education Today 21, 3–17

NMC (2008a) Standards to support learning and assessment in practice. 2nd Edn. London: NMC

NMC (2008b) The Code: Standards of conduct, performance and ethics for nurses and midwives. London: NMC

References

Phillips T., Schostak J., Tyler J. and Allen L. (2000) Practice and Assessment in Nursing and Midwifery: Doing it for Real. English National Board for Nursing, Midwifery & Health Visiting, London

Stuart C C (2007) Assessment, supervision and support in clinical practice 2nd ed London, Churchill Livingstone

Turnock C and Mulholland J (2007) Learning in the Workplace: A toolkit for placement tutors, supervisors, mentors and facilitators. Chichester: Kingsham Press.

References

Watson R, Stimpson A, Topping A and Porock D (2002) Clinical competence assessment in nursing: a systematic review of the literature. Journal of Advanced Nursing 39(5): 421-431.

Additional resource:

www.practicebasedlearning.org/home.htm