promoting excellence in family medicine nmrcgp the clinical skills assessment : an evolving process...

Promoting Excellence in Family Medicine

nMRCGPThe Clinical Skills Assessment :

an evolving process

Prepared by the CSA Operations Group nMRCGP

Promoting Excellence in

Family Medicine

Health Warning

Contemporaryas of May 2007

Promoting Excellence in Family Medicine


Family Medicine

Principles of Assessment (from PMETB)

Principle 1The assessment system must be fit for a range of purposes

Principle 2The content of the assessment will be based on curricula for postgraduate training which themselves are referenced to all of the areas of Good Medical Practice

Principle 3The methods used within the programme will be selected in the light of the purpose and content of that component of the assessment framework.

Principle 4 The methods used to set standards for classification of trainee’s performance / competence must be transparent and in the public domain


Family Medicine

Principles of Assessment (from PMETB) continued

Principle 5 Assessments must provide relevant feedback

Principle 6Assessors / examiners will be recruited against criteria for performing the tasks they undertake

Principle 7There will be Lay input in the development of assessment

Principle 8Documentation will be standardised and accessible nationally.

Principle 9There will be resources sufficient to support assessment


Family Medicine

The MRCGP Curriculum Statements (Principle 2)

Where to find themRCGP website

What are they?Series of statements, each covering different clinical and practice management areas, based on European Academy of Teachers in General Practice (EURACT) framework and Good Medical Practice (GMC document)

Written by a variety of experts in their field, coordinated by RCGP


Family Medicine

The MRCGP Curriculum Statements (Principle 2) continued

How they are being used

Curriculum statements subdivided into ‘Intended Learning Outcomes’. Cases linked to specified learning outcomes within specified curriculum statements.

This enables sampling from across the curriculum, as cases can be mapped to the curriculum statements (or nMRCGP blueprint)


Family Medicine

Components of the nMRCGP

The Applied Knowledge Test

Computer-based test of knowledge using multiple-choice questions, completed on-screen in dedicated Pearson Vue driving test centres around the country.

The Workplace Based Assessment

Formative and summative; variety of measures; based on a series of reviews; electronic portfolio. Tests trainee in his/her place of work, doing what he/she actually does.

The Clinical Skills Assessment

Clinical consulting skills examination, based on cases from general practice, with role players as ‘patients’, and experienced assessors; provides a pre-determined, standardised level of challenge to candidates.


Family Medicine

Why a Clinical Skills Assessment?

Criticism of current MRCGP that there is no clinical consulting skills component

Provides external validation / triangulation with the other testing methods used

Using simulated patients is a validated and reliable method for testing clinical skills, so long as quality assurance of case production, role player and assessor training is carried out.

Able to offer a standardised, pre-determined level of challenge to candidates and to vary this level of challenge as needed by the assessment requirements


Family Medicine

Purpose of the CSA :

‘An assessment of a doctor’s ability to integrate and apply appropriate clinical, professional, communication

and practical skills in general practice’

Integrative skills assessment - tests a doctor’s abilities to gather information and apply learned

understanding of disease processes and person-centred care appropriately in a standardised

context, making evidence-based decisions, and communicating effectively with patients and

colleagues.


Family Medicine

Series of Integrated Developments

Curriculum statements form the basis for the nMRCGP blueprint :

Intended learning outcomes - track from curriculum statements to cases via CSA blueprint and case selection blueprint

Competency progression

CSA Blueprint – based on nMRCGP blueprint, those sections that can be tested by this methodology


Family Medicine

Series of Integrated Developments continued

Case writing proforma that guides case writers through complexities of linking their case, focussing on the ‘nub’ of the case, and writing a marking schedule that reflects this ‘nub’.

Linking the case with searchable keywords so that a defined selection of cases can be found for each time the assessment is run.


Family Medicine

CSA Blueprint derived from the Curriculum

Blueprint area Descriptor

Data gathering and interpretation

Gathering of data for clinical judgement, choice of examination, investigations and their interpretations

Management Recognition and management of common medical conditions in primary care. Demonstrates flexible and structured approach to decision making

Co-morbidity & health promotion

Demonstrating ability to deal with multiple complaints and co-morbidity and to promote positive approach to health

Person-centred approach

Use of recognised communication techniques that enhance understanding of a patient’s illness and promote a shared approach to managing problems

Professional attitude

Practising ethically with respect for equality and diversity in line with accepted codes of professional conduct

Technical skills Demonstrating proficiency in performing physical examinations and using diagnostic and therapeutic instruments


Family Medicine

Case Selection Blueprint

Clinical Skills AssessmentCase Selection Blueprint

Primary nature of case

Primary system or area of disease

Acute Illness

Chronic Illness

Undifferentiated Illness

Psychol and Social

Preventive/lifestyle

Other

Cardiovascular

Respiratory

Neurological/ Psychiatric

Musculo-skeletal

Endocrine/ Oncological

Eye/ ENT/ Skin

Men/ Women/ Sexual Health

Renal/ urological

Gastro-intestinal

Infectious diseases

ETC


Family Medicine

How does the CSA differ from the Simulated Surgery?

Not just a test of communication skills in a clinical setting

Based on the nMRCGP blueprint, and samples across this blueprint.

Will be taken by many more candidates (3,000 - 4,000 per year)

Assesses integrative clinical skills in primary care settings

Candidates will be expected to undertake focussed physical examinations on some of the role players and be assessed on it


Family Medicine

What is the CSA likely to look like?

Starts October 2007

Temporary assessment centre to be used initially in Croydon

Dedicated assessment centre within new College build planned to be ready over the next 3-5 years

Will take place for a number of weeks, several times a year

Will use multiple circuits

Candidate stays in ‘surgery’ and patient and examiner move around circuit


Family Medicine

What is the CSA likely to look like? continued

Will consist mostly of simulated patient cases.13 stations, probably each of 10 minutes

Marks collected by Opscan techniquesSome triangulation with Workplace Based Assessment competenciesStations picked from intended learning outcomes (ILOs) across the nMRCGP blueprint with clear derivation


Family Medicine


Family Medicine

The Marking Schedule

Each case is marked in 3 domains :

Data gathering, examination and clinical assessment skills

Clinical management skills

Interpersonal skills

All domains have equal weighting


Family Medicine

The Marking Schedule continued

Assessor uses word pictures to help decide grade for each domain, then uses this information to make a judgement on the grade for the case overall (4 decisions)

Feedback to candidates

Serious concerns box


Family Medicine

Four possible Grades

Clear Pass (CP)

Marginal Pass (MP)

Marginal Fail (MF)

Clear Fail (CF)


Family Medicine

Three domains for each case :

3. INTERPERSONAL SKILLS:

Demonstrating the use of recognised communication techniques to understand the patient’s illness experience and develop a shared approach to managing problems. Practising ethically with respect for equality and diversity, in line with the accepted codes of professional conduct. (Blueprint: Person-centred approach, Attitudinal aspects).

1. DATA-GATHERING, TECHNICAL & ASSESSMENT SKILLS :

Gathering & using data for clinical judgement, choice of examination, investigations & their interpretation. Demonstrating proficiency in performing physical examinations & using diagnostic and therapeutic instruments. (Blueprint: Problem-solving skills, Technical Skills)

2. CLINICAL MANAGEMENT SKILLS :

Recognition & management of common medical conditions in primary care. Demonstrating a structured & flexible approach to decision-making. Demonstrating the ability to deal with multiple complaints and co-morbidity. Demonstrating the ability to promote a positive approach to health. (Blueprint: Primary Care Management, Comprehensive approach)


Family Medicine

How the CSA is aiming to meet PMETB assessment criteria

PMETB ASSESSMENT CRITERIA

1. This methodology is judged to be the best way to test Clinical Skills in general practice currently.

2. Cases are based on the nMRCGP curriculum.

3. Assessment methodology chosen is fit for purpose. This has been validated and reliable, both elsewhere and by our main pilot.

4. Standard setting will be transparent and in the public domain with wide consultation.

5. Feedback will be given to all candidates.


Family Medicine

How the CSA is aiming to meet PMETB assessment criteria : continued

PMETB ASSESSMENT CRITERIA

6. Recruitment of assessors will be on ability – to rank order, mark reliably, knowledge base.

7. Lay input has been consistently sought.

8. Documentation will be accessible nationally through the College website and publication in peer reviewed journals and the various weekly publications.

9. Resources? Continually under review…


Family Medicine

Further resources

CSA sample cases to be made available soon

Latest information on RCGP website

Introduction to CSA cases document


Family Medicine

Acknowledgements

This presentation written by

Kamila HawthorneNeil Munro

On behalf of the CSA Operations group, nMRCGP

Updated May 2007


Family Medicine

Members of the CSA Operations Group

Adrian Freeman GP, FRCGP Examiner, Chair of the Operations GroupMei Ling Denney GP, MRCGP Examiner and Course Organiser, Deputy Convenor MRCGP Simulated Surgery Kamila Hawthorne GP and Trainer, MRCGP Examiner, Senior Lecturer in General Practice, Cardiff.Fiona Patterson Professor of Work Psychology at City UniversitySue Rendel GP, MRCGP Examiner and Convenor MRCGP Simulated SurgeryAmar Rughani GP, MRCGP Examiner, nMRCGP blueprint guardianDavid Sales GP, Assessment Fellow, RCGPRichard Wakeford MA Cpsychol, FRCGP, Assessment Consultant, University of Cambridge.Faye Geoghegan nMRCGP Project Manager

promoting excellence in family medicine nmrcgp the clinical skills assessment : an evolving process...

Documents

assessment slide

family medicine nmrcgp

assessment framework

pmetb principle

assessment formative

assessment system

family medicine components

nmrcgp blueprint slide