national external quality assessment (eqa) scheme for ... · 3 cervical screening programme -...
TRANSCRIPT
Cervical Screening Programme National external quality assessment (EQA) scheme for gynaecological cytopathology
Ninth edition, March 2018 Public Health England leads the NHS Screening Programmes
2
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
About Public Health England
Public Health England exists to protect and improve the nation’s health and wellbeing,
and reduce health inequalities. We do this through world-leading science, knowledge
and intelligence, advocacy, partnerships and the delivery of specialist public health
services. We are an executive agency of the Department of Health and Social Care,
and a distinct delivery organisation with operational autonomy. We provide government,
local government, the NHS, Parliament, industry and the public with evidence-based
professional, scientific and delivery expertise and support.
Public Health England, Wellington House, 133-155 Waterloo Road, London SE1 8UG
Tel: 020 7654 8000 www.gov.uk/phe
Twitter: @PHE_uk Facebook: www.facebook.com/PublicHealthEngland
About PHE Screening
Screening identifies apparently healthy people who may be at increased risk of a disease
or condition, enabling earlier treatment or better informed decisions. National population
screening programmes are implemented in the NHS on the advice of the UK National
Screening Committee (UK NSC), which makes independent, evidence-based
recommendations to ministers in the 4 UK countries. The Screening Quality Assurance
Service ensures programmes are safe and effective by checking that national standards
are met. PHE leads the NHS Screening Programmes and hosts the UK NSC secretariat.
www.gov.uk/phe/screening
Twitter: @PHE_Screening Blog: phescreening.blog.gov.uk
Prepared by: Sharon Whitehurst
For queries relating to this document, please contact: [email protected]
© Crown copyright 2018
You may re-use this information (excluding logos) free of charge in any format or
medium, under the terms of the Open Government Licence v3.0. To view this licence,
visit OGL. Where we have identified any third party copyright information you will need
to obtain permission from the copyright holders concerned.
Published: February 2018
PHE publications PHE supports the UN
gateway number: 2017796 Sustainable Development Goals
3
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Document information
Only the website version of this document is controlled and located at
www.gov.uk/government/collections/cervical-screening-professional-guidance.
Any downloaded or electronically transmitted file is not a controlled document and it is
the responsibility of individuals to make sure that they are accessing the most recent
version.
The EQA Management Board has approved this document (January 2018).
UKAS accredited Proficiency Testing Provider number 7597.
Accreditation schedule.
4
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Contents
About Public Health England 2
About PHE Screening 2
Document information 3
Contents 4
1. Introduction 7
2. Summary of changes from eighth edition 7
3. Joint Working Group for Quality Assessment in Pathology 9
4. Scope 10
5. Aims 10
6. General information 11
6.1. Scheme provider 11
7. Scheme enquiries 11
8. Organisation and management 12
8.1. Host organisation 12
8.2. Staffing 12 8.3. Governance 14
9. EQA agreement 17
9.1. General terms and conditions of participation 17 9.2. Eligibility requirements 18
9.3. Enrolment 20 9.4. Participant responsibilities 20
9.5. Confidentiality 21 9.6. Subcontracting laboratories 22
9.7. Review of submitted slides 26 9.8. Professional code of conduct 27 9.9. Financial aspects 28
10. Preparation for the EQA round 29
10.1. Organising transfer of slides 29 10.2. Tracking the location of slides 29 10.3. Slide bank 29
10.4. Slide breakage, damage or loss 30 10.5. Assembly of slide sets 30
10.6. Information and instructions for participants 31 10.7. Other documentation 32
11. Conditions for examining EQA slides 34
11.1. Non-medical staff 34 11.2. Consultant biomedical scientists in cervical cytology 35 11.3. Medical staff 36
5
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
11.4. Trainee staff 37 11.5. Additional slide sets 37
12. The assessment 38
12.1. Recording and submitting responses 38
12.2. Following assessment 40 12.3. Reports 41 12.4. Certificates of participation 41 12.5. CPD and CME credits 41
13. Slide consensus 42
13.1. Consensus opinion 42
13.2. Data handling and calculations 42 13.3. Slides with consensus different to the submitted classification 42 13.4. Consensus on negative, inadequate and normal slides 43 13.5. Identifying missed dyskaryosis 44
13.6. Grading consensus for abnormal slides 44 13.7. Slides not reaching consensus 45 13.8. Slides with abnormal consensus but no grade consensus 45
14. Scoring 47
14.1. Consensus slides 47 14.2. Cytology screeners and checkers 47
14.3. Pathologists and consultant biomedical scientists 48 14.4. Transcription errors 49
14.5. Final scores 49
15. Assessment of performance 50
15.1. Cytology screeners and checkers 50
15.2. Pathologists and consultant biomedical scientists 50 15.3. Identification of substandard performance 50 15.4. Persistent substandard performance 52
15.5. Action points for persistent substandard performance 53
16. Communication to and from the EQA scheme 55
16.1. Constructive criticism, comments and compliments 55
16.2. Formal complaints 55 16.3. Formal appeals 55 16.4. Protocol changes 56 16.5. Participants’ meetings 56 16.6. Audit 56 16.7. Surveys 57 16.8. Annual report 58
References 59
Appendix 1: procedure for reporting and investigating alleged collusion 60
Appendix 2: EQA timeline 63
Appendix 3: referral sheet 64
Appendix 4: scoring matrix for cytology screeners/checkers 65
Appendix 5: scoring matrix for pathologists/consultant biomedical scientists 66
6
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Appendix 6: table of performance examples with action points 67
Appendix 7.1: flowchart of actions taken when first action point is reached 68
Appendix 7.2: flowchart of actions taken when second action point is reached 69
Appendix 8: formal complaints procedure 70
Appendix 9: appeals procedure 71
7
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
1. Introduction
This document provides guidance on the scheme design, interpretation of results and
the general requirements and responsibilities for laboratories and participants. It also
sets out the principles and methodology of operation for a consistent delivery across
the nation.
2. Summary of changes from eighth
edition
Section
Sub-section
Summary
8 8.3.1 Clarified that the EQA management board responsibilities are documented in the terms of reference for that committee.
9 9.1 Sentence inserted to confirm that all scheme documents, electronic and paper, are retained in accordance with ‘The retention and storage of pathological records and specimens’, fifth edition April 2015.
9.1.1 Clarified that Royal Mail Special Delivery is the method of approved transport. Deleted sentence, which referred to ‘inter-laboratory’ services and international couriers.
9.5 Clarified that laboratory managers must have systems in place to monitor and review on-going staff participation and performance in EQA, to monitor trends in results as appropriate and recommended that this be formally recorded within the departmental appraisal process. Deleted previous sentence…’participants will be asked to disclose their individual results to laboratory management’.
9.6.2 Clarified that only copy slides must be submitted for inclusion in the EQA rounds.
9.6.7 Clarified that where original material is submitted for EQA then this contravenes the scheme protocol.
8
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9.8 This section was revised to describe the intentions of the scheme in respect of potential collusion.
12 12.1 Paragraph 9, heading changed to ‘Recording and submitting results’. Clarified that participants must upload their results immediately after screening their EQA slides and while the slides are still in the laboratory.
16 16.4 Sentence inserted to clarify that the EQA facilitator notifies participants about agreed changes to the scheme and when they come into effect.
Appendix 1 Revised to describe the process for reporting and investigating alleged collusion.
9
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
3. Joint Working Group for Quality
Assessment in Pathology
The Joint Working Group for Quality Assessment in Pathology (JWG) is a committee of
the Royal College of Pathologists (RCPath), reporting to the Professional Performance
Panel. The JWG is responsible for the oversight of all pathology external quality
assessment (EQA) in the UK, including the approval and registering of schemes, and
the setting of policy and maintenance of appropriate professional standards. As part of
this remit, the JWG monitors the EQA performance of clinical laboratories in the UK.
JWG uses discipline-specific panels, known as National Quality Assurance Advisory
Panels (NQAAP), which report those laboratories that have failed to rectify quality
problems to the JWG.
The JWG subsequently works with failing laboratories to improve standards and has
the responsibility to report persistent poor performance to the Care Quality Commission
and UKAS. Figure 1 outlines the pathway for pathology EQA in the UK.
Figure 1. Pathology EQA in the UK
10
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
4. Scope
The NHS Cervical Screening Programme (NHSCSP) External Quality Assessment
Scheme for Gynaecological Cytopathology and the guidance set out in this document
applies to NHS cervical screening laboratories in England and Wales, and laboratories
outside the United Kingdom which are signed up to the EQA agreement.
5. Aims
The scheme aims to provide an acceptable degree of reliability and consistency by
educating, advising and supporting all participants.
The scheme is based on the assessment of slide sets by all staff who report cervical
cytology on behalf of the NHSCSP. Individual reporting is compared to consensus
reporting to determine whether an individual’s performance falls within an acceptable
range.
There are 2 rounds (2 circulations of slide sets) in each EQA year. An EQA year runs
from 1 April to 31 March.
There are 10 slides in each slide set per round, which are used for performance
analysis.
Up to 2 additional ‘special interest’ slides may be circulated with the set. These are
clearly identified as being for educational interest, and are not included in the
calculation of final assessment results.
11
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
6. General information
6.1. Scheme provider
The provider organisation is a Public Health England (PHE) executive agency of the
Department of Health for its Young Person and Adult Screening Programmes.
Postal address:
Young Person and Adult Screening Programmes (YPASPs)
Public Health England
Level 2, Vulcan House Steel
6 Millsands
Sheffield
S3 8NH
Corporate information: www.gov.uk/PHE
7. Scheme enquiries
All general enquiries must be made directly to the National EQA Team:
National EQA Team
Screening QA Service (Midlands & East) 1st Floor
5 St Philip’s Place
Birmingham
B3 2PW
Tel: (0121) 214 9130
Email: [email protected]
The telephone line and generic email account are manned on weekdays during normal
office hours (9am to 5pm). Staff with dedicated responsibility for EQA services deal with
all enquiries.
12
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
8. Organisation and management
8.1. Host organisation
The Screening Quality Assurance (QA) Service (Midlands & East) hosts and operates
the EQA scheme.
The head of screening QA service (Midlands and East) has national responsibility for
external quality assessment as part of their role for cervical screening. This includes:
day to day support and advice to the EQA operations team
line management for the EQA facilitator and administrative function
responsibility for EQA online through the line management of the SQAS staff
who develop and maintain the system
joint authorisation, with the EQA facilitator, of individual end of round reports
review and authorisation of all operational work instructions and associated
documentation
chairs the EQA operational group meetings
A dedicated EQA manager and administrative team operate the day-to-day
organisation and management of the scheme.
8.2. Staffing
8.2.1. Scheme Organiser
PHE subcontracts the scheme organiser and deputy scheme organiser roles.
The scheme organiser and deputy:
is a consultant cytopathologist, a consultant histopathologist with an interest in
cervical cytopathology or a consultant biomedical scientist who holds the
advanced specialist diploma in cervical cytology
is professionally regulated through their registration body
is a participant in the scheme and has experience of selecting and reviewing
slides for EQA
holds a senior post in a laboratory providing gynaecological cytopathology for
the NHSCSP
participates in an appropriate CPD/CME scheme, and fulfils NHSCSP and pre-
existing professional body requirements for continuing professional development
has some knowledge of quality management systems and accreditation
requirements
13
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
may also be the chair of the scheme’s EQA steering group
The scheme organiser and his or her deputy:
collaborates with the provider operation in setting annual objectives for the EQA
scheme and its strategic direction
receives advice from the EQA steering group on the practical aspects of scheme
design and operation
is responsible for the oversight of poor performance and liaises with NQAAP in
the event of persistent substandard performance
oversees difficult and sensitive situations associated with participant poor
performance, complaints and appeals
approves the scheme’s annual report(s) for service users and management
promotes the value of the EQA scheme and its activities at relevant professional
meetings
promotes and provides advice on the educational elements of the scheme
responds urgently to any situation arising during the EQA round which has the
potential to adversely affect a patient’s management
8.2.2. EQA facilitator
The national external quality assessment (EQA) facilitator is employed by PHE. This
individual has the authority to manage the day-to-day operation of the scheme in line
with published protocols and guidelines. This includes responsibility for the
management and operation of the quality management system. The EQA facilitator
reports to the scheme organiser in a professional capacity, and is accountable to the
head of screening quality assurance (Midlands and East) who is also their direct line
manager.
The EQA facilitator is responsible for:
day to day management of the EQA scheme
development and maintenance of local administrative and quality systems in line
with published protocols and quality procedures
undertaking the role of scheme quality manager
overseeing arrangements for giving laboratories and participants appropriate
notice of EQA requirements
identifying and highlighting actual or potential issues in relation to the scheme’s
operation and bringing these to the attention of the appropriate manager
maintaining all aspects of confidentiality and for the safe storage, recording and
tracking of samples submitted for use in EQA
working closely with the local support team to identify areas for development in
relation to the online EQA reporting system
14
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
preparing the audit strategy and internal audit programme and making sure
findings are recorded and completed appropriately
preparing for external audits and maintenance of routine activities in line with
UKAS requirements
producing accurate reports as requested for management, steering group and
operational review meetings
taking action in the event of poor performance as documented in this protocol,
and act as the scheme secretary as required
The EQA facilitator is supported by a local administration team hereafter termed
national EQA team. The head of screening quality assurance and 2 administrative
assistants provide cover for the EQA facilitator in his or her absence, appropriate to the
cover required.
The national EQA team comprises staff with defined responsibilities to support the EQA
facilitator as well as responsibilities for management and IT/data support. The cytology
education manager (under contract to PHE Screening) maintains the national scheme
protocol. Appropriate cover is provided in his or her absence.
8.3. Governance
Figure 2 below illustrates the EQA relationships within the scheme.
8.3.1. EQA management board
The EQA management board is an executive team which provides a review and
governance mechanism for the scheme. The EQA Management Board terms of
reference provide clear details of its roles and responsibilities.
The EQA management board acts on behalf of and has direct access to highest-level
management Membership includes: national cervical screening programme manager,
scheme organiser, deputy organiser, EQA facilitator and head of screening quality
assurance (Midlands and East).
The national cervical screening programme manager has oversight of the EQA scheme
and reports directly to the national programmes lead, young person and adult screening
programmes. The national cervical screening programme manager does not have a
role in operational delivery and is chair of the EQA management board. The meetings
are administered by the national EQA team.
The EQA management board is responsible for the strategic direction, planning and
development of the scheme. The EQA management board has considered all the
essential requirements of planning, including:
15
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
statistical design
suitability of test items
frequency of assessment and evaluation criteria
operational activity
quality management system
resources required to implement the processes
number of participants
cross-boundary working
8.3.2. EQA steering group
The EQA steering group is set up to provide a supporting and advisory role and meets
on a quarterly basis. Members include individuals from participating laboratories to
make sure the requirements and interests of participants are represented. The EQA
steering group provides technical advice and assistance on scheme design and
operational issues. Its role does not extend to involvement in performance issues
unless these are related to the scheme’s design and performance monitoring systems.
The scheme organiser chairs the meetings. The meetings are administered by the
national EQA team.
8.3.3. Operational group
The operational group is set up to discuss all operational activity and meets on a
quarterly basis. Members include the national EQA administration team, staff
responsible for the maintenance of EQA online and a representative from the national
programme team responsible for protocol maintenance. The scheme organiser or
deputy organiser may attend to provide advice (as necessary). The Head of Screening
QA Services (Midlands and East) chairs the meetings. The meetings are administered
by the national EQA team.
8.3.4. Online web-based service for participants
The nationwide transition to online data entry and results management was made on 1
April 2015. The bespoke EQA website (hereafter termed ‘EQA online’) is secure and
access is via an N3 connection.
EQA online is managed by the national EQA team in Birmingham and provides a
secure facility for existing participants to enter their results for the current round, and to
view, download and/or print their individual personal reports.
Suitable provision is made for individuals who are unable to access the website through
the use of manual recording and data entry systems. Laboratories without an N3
connection can scan and email their EQA response forms securely.
16
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Access to this service is achieved through the participant registration and sign up
process.
An email facility is provided for account queries including lost or forgotten passwords
and login details.
Figure 2. Inter-relationships within the EQA scheme
DIRECTOR OF SCREENING Public Health England
Screening Programmes and Screening QA Service
Young Person and Adult
Screening Programmes
Cervical Screening Programme
SCHEME PROVIDER EQA Management Board
Screening Quality
Assurance Service
SCHEME OPERATION & QUALITY
MANAGEMENT National EQA Team
SCHEME ORGANISER
(Professional Accountability)
EQA STEERING GROUP (Advisory)
PARTICIPANTS/ LABORATORIES
Slide selection and review
17
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9. EQA agreement
There is an agreement between the PHE Screening QA Service and NHS cervical
screening laboratories and their staff who report gynaecological cytopathology for the
NHS Cervical Screening Programme. It is also applicable to private laboratories and
those from outside the UK who participate in the scheme. The general terms and
conditions of participation for all parties are documented in the agreement. The
agreement is subject to periodic review.
9.1. General terms and conditions of participation
Participation in the EQA scheme is mandated for all staff who report gynaecological
cytology for the NHSCSP. This applies to all:
cytology screeners
biomedical scientists
consultant biomedical scientists (also known as advanced BMS practitioners in
cervical cytology)
pathologists
Participation is mandated for all staff whether employed on a full-time, part-time,
permanent or temporary basis.
Participation in relevant accredited EQA schemes is a prerequisite for laboratories
registered with UKAS and seeking accreditation to ISO 15189:2012.2
Laboratories are responsible for notifying the national EQA team of any changes in
service provision, and changes in staffing (whether permanent or temporary) including
new recruits, leavers and locums.
Where possible, the EQA team will make reasonable adjustments to facilitate access to
the scheme for all potential participants. Laboratories are responsible for notifying the
national EQA team if a member of staff has any particular needs, which may require an
adjustment as to how they participate in EQA.
Participants must use the LBC methodology that they routinely report in - ThinPrep or
SurePath. Individuals who routinely report in both ThinPrep and SurePath must
alternate their participation between the 2 technologies by agreement with the national
EQA team. These individuals must not participate in both technologies in a single EQA
round.
18
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Laboratories must confirm their acceptance of all terms and conditions described in the
EQA agreement in order that their staff can enrol on the scheme. There is no
requirement for annual re-enrolment as this is managed through periodic review and
renewal of the EQA agreement.
Laboratories are responsible for the safekeeping of EQA slides during each round.
Laboratories must report damage or breakages to the EQA facilitator immediately via
email. The EQA facilitator advises the laboratory how to proceed in these
circumstances.
Slides must be protected from sunlight when not in use in order to preserve their quality
for subsequent participants.
All data entry and result reports are managed via the EQA website.
All scheme documents, electronic and paper, are retained in accordance with ‘The
retention and storage of pathological records and specimens’, fifth edition April 2015. 5
9.1.1. Labelling, packaging and transport
Laboratories are provided with detailed instructions to make sure that EQA slides are
correctly and safely packaged and labelled in line with Royal Mail requirements. Slides
must be transported by Royal Mail Special Delivery. This method allows packages to
be tracked during transit and upon receipt. Other methods of transport may be used
subject to approval by the scheme’s management. Laboratories must not use
unauthorised methods of slide transport.
9.1.2. Sanctions for non-compliance with the EQA agreement
Laboratories, which do not conform to the protocol requirements, are in breach of the
EQA agreement and may be removed from the scheme. The scheme has established a
formal escalation policy. Removal from the scheme presents a risk to a laboratory
achieving or maintaining ISO 15189:2012 accreditation.
9.2. Eligibility requirements
9.2.1. Students and trainees
Students and trainees who intend to pursue a career that includes cervical cytology,
and who are working in an NHSCSP cervical screening laboratory or a cervical
screening laboratory under contract to the NHS, are eligible to participate in the
scheme for educational purposes only. Scores are excluded from the performance
analysis. Laboratory managers must make sure that this staff group discusses results
with the relevant training officer/supervisor.
19
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9.2.2. Locum staff
Locum staff participate in the national scheme and comply fully with all conditions and
arrangements if they work:
in an NHSCSP cervical screening laboratory
in a cervical screening laboratory under contract to the NHS
Note that production of a satisfactory report and/or evidence of participation in EQA
does not guarantee eligibility for employment in an NHSCSP cervical screening
laboratory or a cervical screening laboratory under contract to the NHS.
9.2.3. Staff working in private laboratories under contract to the NHS
Staff working in private laboratories where screening is undertaken for the NHSCSP
participate in the scheme must comply fully with its conditions and arrangements.
9.2.4. Staff working in private laboratories in the UK
The EQA agreement also describes the conditions of participation for staff working in
private laboratories (non-NHSCSP screening) within the UK, and staff working in
cervical cytology laboratories outside the UK.
Staff working in private (non-NHS) laboratories where non-NHSCSP screening is
performed may wish to subscribe to the scheme. Participation is not mandated for
these individuals.
Individuals working in private laboratories who choose to participate in the scheme
must note that their responses are not included in the consensus calculations to
determine slide classifications. In addition, their overall scores are not included in the
statistical analysis used to determine individual performance.
Results and feedback to these participants are determined by comparison against
participants who do perform screening work for the NHSCSP.
9.2.5. Staff working in laboratories outside the UK
Subject to operational capability and resources, the scheme is open to laboratories
from outside the UK subject to their meeting specified terms and conditions of the EQA
agreement.
Participants working in laboratories outside the UK must note that their responses are
not included in the consensus calculations to determine slide classifications. In addition,
20
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
their overall scores are not included in the statistical analysis used to determine
individual performance.
Non-UK laboratories declare the pathway for professional accountability within their
employing organisation and regulatory body under the terms and conditions of the EQA
agreement. Persistent poor performance is reported to the relevant regulatory body.
9.2.6. Ineligible staff
Individuals are not eligible to participate in the EQA scheme if they are not currently
screening and reporting.
9.3. Enrolment
On enrolment, participants are directed to the secure EQA website for general
guidance and instructions on how to set up and manage their EQA account.
Participants must use an employer issued email address to set up and operate an EQA
account. Participants unable to access EQA online using an N3 connection will be
advised by the national EQA team of the alternative arrangements for participation
using established manual systems.
Participants will be asked to confirm they have read and understood this document.
Participants who are unsure about any EQA website content or a scheme operational
matter must contact the EQA facilitator for guidance.
9.4. Participant responsibilities
9.4.1. Late participation
Participants must be available to complete the EQA during the time the slides are
present in the laboratory.
Provision can be made for staff who are unavailable on the agreed EQA dates for a
legitimate reason (for example due to pre-arranged annual leave or a short-term
authorised absence). These individuals may participate at a neighbouring laboratory, or
slides can be made available on an alternative date at the end of the round or at
another suitable point in the cycle (only one further date will be offered). Local
management is responsible for making sure that staff are allocated time away from the
laboratory to participate in EQA elsewhere if necessary.
21
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9.4.2. Non-participation
Participants need to be aware of the consequences of non-participation in EQA.
An individual who is absent on the agreed EQA date, or who does not participate
without a legitimate reason, is classed as a non-participant without a valid reason. This
may lead to a result of poor performance being recorded.
If an individual fails to participate in 2 out of 3 rounds without legitimate reasons, this
information is passed to the:
scheme organiser
head of screening QA
head of screening QA responsible for the relevant laboratory (who alerts the
commissioners of the service)
chair of NQAAP
Legitimate reasons for non-participation include:
short or long-term authorised absence
maternity leave
pre-arranged annual leave
an extended period of sabbatical leave
9.5. Confidentiality
Information disclosure liabilities, responsibilities and procedures are communicated to
and followed by all EQA staff. This includes staff employed directly by and/or under
contract to PHE.
All participant data is confidential. All electronic data is stored in password-protected
files or in areas accessible only to authorised EQA staff. Paper based data is stored in
a secure, locked cabinet and access is restricted and limited to only authorised EQA
staff.
The scheme is confidential under the conditions of participation in EQA schemes
determined by the professional bodies through the Joint Working Group for Quality
Assurance (JWG).
Participants may choose to waive confidentiality for the purposes of discussion and
mutual assistance. They can also choose to waive confidentiality for regulatory or
recognition purposes.
22
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
The identities of participants are not made available except under circumstances stated
in the terms and conditions of participation. The EQA facilitator will only provide
participant names and corresponding codes:
when a participant is enrolled on the scheme
when a participant has requested a reminder of his or her code number either in
writing or e-mail or via the facility provided in their online EQA account
when circumstances dictate that the chair of the JWG on quality assurance is
notified of a member of medical staff or a consultant biomedical scientist who
has reached the second action point (see section 15.5.2). The chair of the JWG
may request that confidentiality is not broken in these circumstances
Laboratory managers must have systems in place to monitor and review on-going staff
participation and performance in EQA, and to monitor trends in results as appropriate. It
is recommended that this is formally recorded within the departmental appraisal
process.
At present, it is considered extremely unlikely that any ‘interested parties’ or
government regulatory authorities require participants’ results to be provided directly to
them.
If in the unlikely event the provider organisation receives a third party request from a
regulatory authority requiring EQA results, the source will be investigated and verified
by the national EQA team. The provider organisation will comply with a valid and
verified request, and notify the affected participants of this action in writing within 5
working days.
9.6. Subcontracting laboratories
PHE subcontracts with the NHSCSP cervical screening laboratories that provide slide
material, and the staff who review it. The provision of suitable slides and qualified staff
who perform the slide reviews is critical to the success and integrity of the scheme.
Cervical screening laboratories not under contract to the NHS (including those outside
the UK) are not quality assured by PHE or its equivalent in other parts of the UK. They
are not eligible therefore to submit slide material or provide staff to undertake slide
reviews since these services are not required to conform to national screening
programme standards and guidelines.
NHSCSP cervical screening laboratories or those under contract to the NHS are
accredited by or registered for accreditation with United Kingdom Accreditation Service
(UKAS). Laboratories that are not yet accredited to ISO 15189:2012 must make sure
that only qualified and competent staff undertake slide selection, submission and slide
review activities for the scheme.
23
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Staff are deemed competent on the basis of:
qualification
experience
satisfactory participation in the scheme
employment by a laboratory which is contracted to provide a cervical screening
service to the NHSCSP
9.6.1. Source of slides
All NHSCSP cervical screening laboratories (or cervical screening laboratories under
contract to the NHS) are required to submit slides under the terms and conditions of the
EQA agreement.
Controls are in place to minimise loss or damage to slides during EQA activity and this
is monitored annually. Any loss or damage to stakeholder property which renders it
unsuitable for use is recorded and reported to the stakeholder. (Stakeholder property is
physical property including glass slides, participant information and scheme data.)
The total number of slides requested for any particular round is at least 3 times the
number that will be needed. This is because up to two-thirds of slides submitted may be
unsuitable due to poor technical quality or a lack of agreement on the classification of
the slide.
‘Spoke’ laboratories for liquid based cytology (LBC) processing arrangements are also
eligible to submit slides, provided that both the ‘hub’ and ‘spoke’ laboratories meet the
terms and conditions described in the EQA agreement.
Laboratories must keep a record of the slides and supporting documents submitted for
EQA purposes.
The national EQA team returns all slides to the submitting laboratory at the end of the
EQA year (or sooner) by arrangement.
The national EQA team provides feedback to submitting laboratories on panel
consensus and laboratories are required to consider this information during their review
and EQA performance monitoring processes.
9.6.2. Copy slides
Only copy slides must be submitted to the scheme for inclusion in EQA rounds. Copy
slides (repeat preparations) produced from a single LBC sample are requested from
each laboratory. If the laboratory produces more than one copy from the same sample
24
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
then each is considered as an entirely separate case for EQA purposes. Identical
consensus for each of the copy slides must not be assumed.
9.6.3. Slides selected for EQA
Laboratories must provide good (not necessarily easy) examples for potential use in
EQA.
The slides selected for EQA range across the classifications recognised by the British
Association for Cytopathology (BAC). The following classifications are covered:
inadequate
negative
borderline change in squamous cells
borderline change in endocervical cells
low-grade dyskaryosis
high-grade dyskaryosis (moderate)
high-grade dyskaryosis (severe)
invasive squamous carcinoma
glandular neoplasia of endocervical type
glandular neoplasia (non-cervical)
For further explanation see ‘Achievable Standards, Benchmarks for Reporting and
Criteria for Evaluating Cervical Cytopathology’.3
Slides containing infections may be included, although there is no requirement to record
these on the response form. Interesting cases may be included to add educational
value to the scheme. Rarities are not included, nor are slides from unusual or problem
cases unless deemed ‘difficult and clear cut’.
Slides must have histological confirmation of diagnosis if they show:
high-grade dyskaryosis (moderate)
high-grade dyskaryosis (severe)
invasive squamous carcinoma
glandular neoplasia of endocervical type
glandular neoplasia (non-cervical)
Slides that are high-grade (moderate) or high-grade (severe) must be histologically
confirmed as either CIN2 or CIN3 (an exact match is unnecessary). Slides showing
borderline change in squamous cells, borderline change in endocervical cells, low-
grade dyskaryosis, or negative cytology should be submitted independently of any
associated HR-HPV test results.
25
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9.6.4. Slides agreed for submission
Submitted slides must have achieved an in-house consensus opinion among staff,
including a cytology screener, a checker and a pathologist/consultant biomedical
scientist.
Staff who can select slides include cytology screeners, checkers, consultant biomedical
scientists and pathologists. These individuals are qualified and trained to report cervical
cytology, and are currently reporting cervical cytology on behalf of the NHSCSP. These
individuals are responsible for assessing the suitability and technical quality of slides for
potential use in EQA.
9.6.5. Supporting information for submitted slides
The laboratory provides the following information with each slide submitted:
the consensus opinion on the classification of each slide, based on agreement
between at least one cytology screener, one checker, and one
pathologist/consultant biomedical scientist
clinical information derived from the HMR101 form or its electronic equivalent
(age of individual and last menstrual period (LMP)/cycle stage)
histological confirmation of diagnosis for any cases with a consensus opinion of
high-grade dyskaryosis (moderate) or worse
Cytology samples showing low-grade dyskaryosis or borderline change in squamous or
endocervical cells may not have had further follow-up, since the woman may have been
returned to routine recall after HPV triage. In such cases, it may not be possible to
obtain histological confirmation, and slides falling into these reporting categories do not
need to be supported by further slides and/or histology.
Negative or inadequate slides do not require a second slide confirming the classification
in order to be included in the EQA scheme.
If copy reports are provided, only the corresponding slide number must be used for
identification. Any patient identifiable information must be removed.
26
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
9.6.6. Slide confidentiality
The laboratory is responsible for concealing any patient identifiable information on the
slides before submission. The laboratory must obscure the names of the patient and
the submitting laboratory for the purposes of the EQA round but not remove them from
the slide. The laboratory must make sure that the slides are sufficiently labelled to allow
the EQA facilitator to match slides to the relevant accompanying information.
9.6.7. Recording of dots or markers
Where a laboratory submits original material, in contravention with the scheme protocol
it is responsible for keeping a record of the coordinates of any dots or markers that
have been placed on the slides during screening or reporting. This is for medico-legal
reasons. The laboratory is also responsible for removing these dots or markers prior to
submission of the slide.
9.6.8. Dotted templates
EQA does not permit the preparation of dotted templates for staff on ‘catch-up’ prior to
their participation.
9.7. Review of submitted slides
Slides submitted by laboratories undergo a further review before they are included in
the scheme. The EQA facilitator is responsible for organising the slide review.
9.7.1. Reviewers
Laboratories are responsible for providing qualified and competent staff to undertake
the slide review. A cytology screener, a checker and a pathologist/consultant
biomedical scientist undertake the review.
Slide reviewers can report slides, which they may have examined as part of the review
process. To reduce the risk of recognition, slides are anonymised for the review
process, and anonymised again if used in the EQA round. Slide sets generated for
EQA, as far as possible, minimise the number of slides included from a single
laboratory.
9.7.2. Review method
The reviewers agree the classification of a slide that is consistent with the submitting
laboratory opinion, patient history, or histology where appropriate (for example a
cytological report of high-grade dyskaryosis (moderate) matching a histological
diagnosis of CIN II). The panel reaches its consensus without prior knowledge of the
submitting laboratory’s classification.
27
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Slides accepted for EQA are assessed for their technical quality, including stain quality,
mounting and any cracks. Slides that are not technically adequate are rejected.
9.7.3. Discordant opinions
If the panel opinion differs from the submitted opinion and would result in the woman
being managed differently, the EQA facilitator notifies the originating laboratory and
returns the slide(s) immediately. For example, there may be disagreement on whether
to report the slide as negative, inadequate, or abnormal. Alternatively, the agreed panel
opinion may be that the slide should be reported as high-grade (moderate) or worse,
whereas the laboratory may have reported the slide as borderline change in squamous
or endocervical cells, or low-grade dyskaryosis.
It is the responsibility of the medical head of the submitting laboratory to take any
necessary follow up action in the event of such notification.
In the rare event that a participant discovers any unexpected clinical findings (for
example, psammoma bodies) during an EQA round, they must bring this to the
attention of the EQA facilitator in the first instance. The EQA facilitator will speak with
the scheme organiser to confirm the findings and ascertain the required action. The
scheme organiser will contact the submitting laboratory immediately to agree the
appropriate action. Follow up regarding any clinical implications for the woman is the
responsibility of the submitting trust.
9.8. Professional code of conduct
The scheme relies on each participant’s professional honesty to make sure that cases
are not discussed until all staff in the laboratory (excluding absentees) have completed
and submitted their EQA responses.
Participants must not pass on information during the EQA round to allow his or her
opinions to be submitted by another. The scheme defines such behaviour as collusion.
Collusion is a behaviour, which could compromise the validity of results and
consequently undermine the scheme’s integrity, as well as damage an individual’s
professional integrity.
At the beginning of every EQA round, laboratory managers must highlight the potential
consequences of collusion:
any evaluation of personal performance is meaningless
it allows an individual(s) to gain unfair advantage over his or her colleagues
the risk of submitting false responses is increased and this could adversely
affect individual performance
28
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
whether proven or unproven, collusion seriously diminishes an individual’s
professional integrity
the value and integrity of the EQA scheme is undermined
Laboratory managers must be sensitive when dealing with poor performance and
provide support for underperforming staff. Underperforming staff must avail themselves
of the support provided by management.
These measures are in the interests of all participants with the aim of avoiding adverse
effects on individual performance.
The EQA facilitator must be notified, in the first instance, if there is evidence of or a
strong suspicion of collusion. The scheme organiser will collaborate with the relevant
laboratory. The laboratory is responsible for investigating the allegation at local level
and reporting to the scheme organiser. If collusion is proven or admitted, then the
matter will be escalated immediately. Notification will be sent to NQAAP, the employer,
SQAS, and the relevant regulatory organisation.
The procedure for reporting and investigating alleged collusion is described in
Appendix 1.
9.9. Financial aspects
The funding of the scheme is included in the budget of the screening division of PHE.
Employing laboratories must meet costs incurred by individuals needing to travel to
another location to participate in EQA (such as absentees or locum staff).
The EQA scheme is free of charge for laboratories in England providing a cervical
screening service to the NHSCSP. A charge is made for providing the EQA scheme to
NHS laboratories outside England and to private laboratories that are not contracted to
the NHSCSP.
29
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
10. Preparation for the EQA round
Participants are provided with detailed information about the scheme, including their
rights and responsibilities, in advance of the EQA round. This information is provided
via the web-based EQA system or email if the system is not available to them.
10.1. Organising transfer of slides
The EQA facilitator is responsible for working with laboratories to agree dates for slide
receipt, onward transmission or return of slides, and a closing date and time for
submission of responses. See Appendix 2 for the timeline for the EQA schedule.
10.2. Tracking the location of slides
The EQA facilitator is responsible for knowing the location of each and every slide
during an EQA round, and ensuring that this information is documented.
10.3. Slide bank
The review panel(s) selects slides used in the EQA rounds. The EQA facilitator is
responsible for establishing a comprehensive slide bank record and its maintenance.
10.3.1. Obscuring laboratory details
The EQA facilitator must make sure that any laboratory details that would disclose the
origin of a slide are obscured for the duration of a round. This information is not
removed from the slide, but covered by a label that only shows the unique identifying
number of the slide and set.
10.3.2. Removal of dots/markers
The EQA facilitator must check that all dots/markers have been removed from
submitted slides.
10.3.3. Preservation of staining quality
Within each new round, the EQA facilitator must vary the order in which slide sets are
circulated between laboratories. This will make the distribution of pale slides more
equitable if staining deteriorates as a result of repeated microscopy during the EQA
round. Laboratories must protect the slides from sunlight in order to preserve the quality
of the slide for subsequent participants. Participants are provided with instructions on
handling slides, their preservation during EQA and storage prior to their return.
30
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
10.3.4. Slides used in more than one round
If EQA slides are to be used for a third round, then they must be reassessed
beforehand to make sure that their technical quality has not diminished. Slides retained
for future use, or as a contingency measure, must be stored in a suitable light-
impervious container within a locked drawer or cabinet.
10.4. Slide breakage, damage or loss
The laboratory lead must notify the EQA facilitator immediately via email if slides are
reported as broken, damaged or missing either in transit or during the EQA round. The
EQA facilitator will advise the laboratory lead regarding next steps.
The nominated QA biomedical scientist must make an assessment of the slide and
advise if the slide can be repaired, if the damage is not sufficient to disadvantage any
participants, or if the slide must be removed from the circulation. Repairs must not be
attempted without the written consent of the submitting laboratory.
The scheme considers lost or broken (irreparable) slides to be non-consensus and they
are not included in performance calculations. It is impossible to replace damaged or
broken slides with others from the same category, as no two slides are identical. In
such instances, the EQA facilitator notifies the submitting laboratory of the breakage.
At the end of the round, broken or damaged slides are returned to the submitting
laboratory with the rest of the batch. If any slides returned to the national EQA team are
found to be broken upon receipt then repairs are only attempted with the written
consent of the submitting laboratory.
10.5. Assembly of slide sets
The EQA facilitator is responsible for making sure the slide sets are assembled and
numbered accurately.
10.5.1. Number of sets required for cytology screeners/checkers
The EQA facilitator will determine how many slide sets are required to complete the
round. A minimum of 45 participants (excluding trainees and participants working in
laboratories that only screen private work) will see each slide within a set. If a slide
circulation has 200 cytology screeners/checkers, then the maximum number of slide
sets is 5 (4 plus one for the pathologists/consultant biomedical scientists).
The EQA facilitator makes sure that each peer group contains a minimum number of 45
people (preferably more) before the round begins.
31
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
10.5.2. Composition of slide sets
Each slide set contains 10 EQA slides plus any additional slides included for
educational interest. These additional slides are clearly labelled as such, and do not
form part of the assessment. The EQA facilitator determines the final composition of the
slide sets using the following criteria.
1. Slides are selected from all the categories recognised by the BAC. Slide sets
must comprise a greater proportion of abnormal cases than is seen in routine
practice.
2. Each slide set contains between 2 and 6 negative or inadequate slides. The
EQA facilitator must vary the ratio of negative or inadequate slides to slides of
other types for each round.
3. Where more than one slide set is used for a peer group, each slide set must
contain the same number of slides from each classification in each round.
4. All classifications must be covered in any one EQA year (within two consecutive
rounds). This includes negative, inadequate, borderline change (of either type),
low-grade dyskaryosis, high-grade dyskaryosis (moderate), high-grade
dyskaryosis (severe), and wherever possible, ?invasive squamous carcinoma
and ?glandular neoplasia.
10.5.3. Pathologists’ slide set
Slide set(s) allocated for the sole use of pathologists/consultant biomedical scientists
must be clearly labelled as such. They are selected at random from the available sets
and are, ideally, seen by at least 45 participants. In the event that the number of
pathologists/consultant biomedical scientists falls below this number, then a minimum
of 25 participants is acceptable for statistical validity.
10.6. Information and instructions for participants
Participants receive supporting information and instructions for use prior to the start of
an EQA round and the circulation of slide sets. The EQA facilitator instructs participants
on where and how to access the relevant information online.
Supporting information includes:
a response form
a referral sheet
slide set details
participant instructions for recording and submitting responses
relevant information for special interest slides
a complaints procedure
32
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
an appeals procedure
Participants using manual systems will receive paper documentation before the EQA
round starts.
10.6.1. Response form
The paper version of the response form replicates the content of the online version. For
manual systems, each participant receives their response form in a sealed envelope or
by secure email.
10.6.2. Referral sheets
In each round, one or more cytology screeners from each laboratory must mark the
pathologists’ slide set in addition to their EQA. These individuals are required to
complete the referral sheet (Appendix 3). In addition to completing their own EQA, one
or more checkers must assess each slide that the cytology screener considers
potentially abnormal and to complete the standard referral sheet.
The completed referral sheet is circulated to all participants viewing the pathologists’
slide set by the laboratory manager or nominated deputy. The original referral sheet is
retained by the laboratory. There is no requirement to submit this form as part of the
assessment.
10.6.3. Slide set details
Slide information accompanies each slide set. For each slide within the set, the
following information is given:
the slide number
the age of the woman
LMP
HR-HPV test results do not accompany the EQA slides, as this would give an indication
of the cytological classification of the slide. The classification of slides
included for educational interest, with any further comments, accompany the slide set.
10.7. Other documentation
The following areas of guidance are made available to participants in either written or
electronic format, or reference is provided to the location of controlled documents. This
includes instructions and/or action to take regarding:
receipt of broken or damaged slides
slides damaged or broken during the EQA round
33
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
handling, cleaning and storing slides during the EQA round
unexpected findings during a live round
safe packaging and dispatch of slides
34
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
11. Conditions for examining EQA slides
Participants examine EQA slides under conditions, which resemble those used in their
routine practice as closely as possible. They will not undertake the assessment in an
examination setting. Participants are required to evaluate their own diagnostic opinion
against those of their peers. Participants must not discuss the slides in any way before
all participants (excluding absentees) have viewed them and recorded and submitted
their results.
In each round, there are 2 peer groups: cytology screeners and checkers, and
pathologists and consultant biomedical scientists. Where a peer group uses more than
one slide set, participants are each allocated a set. The EQA facilitator makes sure that
the number of participants viewing each set is sufficient for a consensus calculation to
be valid.
The slide categories are representative of the categories of slide result found in routine
workload. The nature and numbers of slides circulated and the results analysis are key
features of the scheme.
11.1. Non-medical staff
In NHSCSP, laboratories there are 2 clearly identifiable tiers of activity undertaken by
non-medical staff: primary screening and checking. At present, staff (other than
consultant biomedical scientists) who are not medically qualified do not report abnormal
slides. They sort slides into those they report as negative or inadequate, and those that
they pass to more senior colleagues for further review. Non-medical staff are assessed
on the basis of their ability to distinguish between negative, inadequate, and abnormal
slides. Many laboratories routinely encourage their checkers to suggest a classification.
The scoring scheme covers all staff in order to enhance the educational benefit of the
scheme.
Individuals (usually cytology screeners and biomedical scientists) who undertake
primary screening decide whether a slide is negative, inadequate, or potentially
abnormal. They refer abnormal slides on to senior staff for more detailed reporting.
Individuals who routinely carry out primary screening undertake EQA with unscreened
and unmarked slides.
There is no limit set on the maximum time that cytology screeners can look at a
particular EQA slide. As assessment conditions resemble those of normal working
practice, the expectation is that cytology screeners are able to screen the 10 EQA
slides in approximately 2 hours.
35
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
A cytology screener is assessed on only one set of slides. Performance is measured on
his or her ability to differentiate between negative, inadequate, and potentially abnormal
slides, and on any instances of missed dyskaryosis. Cytology screeners are required to
suggest a grading classification for abnormal slides, and quantitative and qualitative
feedback based on these opinions is provided for personal educational purposes. This
feedback is not included in the performance analysis.
In addition, cytology screeners can view and report slides used in the EQA scheme for
pathologists/consultant biomedical scientists. Their results are treated as purely
educational and are not used in performance analysis.
Checking
Experienced cytology screeners and biomedical scientists have varied duties. Checkers
usually undertake some primary screening, rapid review, and checking of cervical
slides that appear potentially abnormal. Checkers, like cytology screeners, report one
set of 10 unmarked slides.
There may be some checkers who do not perform any primary screening and who look
only at marked slides. These individuals are allowed to perform their EQA on marked
slides, however they report all 10 slides within the slide set, not only those that have
been marked. Cytology screeners within the laboratory must mark the slides as they
would in routine practice before the checkers perform their EQA.
Checkers who receive marked slides do not receive a copy of cytology screener
opinions. The performance of those checkers who do not engage in primary screening
is calculated in the same way as that of cytology screeners and checkers who do.
Checkers receive quantitative and qualitative feedback for personal educational
purposes on their grading of abnormalities. Assessment of their performance is based
on their ability to distinguish between negative, inadequate, and potentially abnormal
cervical samples and on any instances of missed dyskaryosis.
Checkers can view and report slides used in the EQA scheme for
pathologists/consultant biomedical scientists. Their results are treated as purely
educational and not used in performance analysis.
11.2. Consultant biomedical scientists in cervical cytology
Consultant biomedical scientists sign out abnormal slides and provide management
recommendations. They may also undertake checking duties and report unchecked
slides, effectively acting as their own checker. EQA for these staff uses slides that have
been screened by cytology screeners and other checkers, in line with routine practice in
the laboratory.
36
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
While consultant biomedical scientists act under the direction of the consultant
pathologist, they are not under his or her direct supervision. From an EQA point of
view, it is therefore appropriate that they are included alongside medical practitioners.
There may be some consultant biomedical scientists who perform primary screening.
These individuals are not expected to participate in EQA as a cytology screener.
11.3. Medical staff
The 2 major activities undertaken by most pathologists who routinely report for the
NHSCSP are as follows.
1. Reporting of slides referred from cytology screeners and checkers as potentially
abnormal.
2. Review of slides that have previously been reported as negative or inadequate
by cytology screeners or checkers, and have later been identified as requiring
medical review.
The EQA scheme for pathologists assesses an individual’s performance in providing an
opinion on slides identified as potentially abnormal, and in reviewing negative and
inadequate slides that have been through primary screening. Cytology screeners and
checkers may wish to participate in this type of EQA in addition to their regular EQA
Their responses are not used for performance assessment, and are not included in the
analysis of the slide set.
EQA for medical staff is based on slides that have been screened by cytology
screeners and checkers in line with routine practice in the laboratory. Practice varies
between laboratories, but cytology screeners and checkers often mark slides. Where
this is normal practice, marked slides are retained for EQA purposes. Pathologists and
consultant biomedical scientists know which slides have been referred by cytology
screeners and which by checkers, but not the suggested classifications of the slides
offered by either group.
The response form used by medical staff when reporting slides is in line with
recommendations from the British Association for Cytopathology, in particular their
Code of Practice for Cytopathology Laboratories.4 This states that the pathologist
should see all abnormal material and a proportion of negative material to make sure
that laboratory is maintaining accuracy and quality.
A pathologist has experience of screening unmarked slides (and, in particular, of
rescreening negative slides when abnormalities have subsequently been found), and of
rescreening an entire slide when equivocal cell groups have been identified as requiring
an opinion. Pathologists who undertake EQA assessment examine the whole of every
37
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
slide, including slides that have already been marked by cytology screeners and
checkers.
The cytology screeners selected to screen slides in the pathologists’ set mark these as
they would in routine practice. They indicate (with an ‘R’ on the referral sheet), any
slides that they consider potentially abnormal. The checkers then look at the slides
marked with an ‘R’ and indicate (with an ‘A’ on the referral sheet) any slides that they
would refer to the pathologist/consultant biomedical scientist as potentially abnormal.
The checkers also mark the slides as they would in routine practice.
The pathologists and consultant biomedical scientists then perform their EQA on the
marked slide set using only the referral sheet for reference (they remain unaware of
cytology screener/checker opinions concerning the classification of the slides). All EQA
slides within the set are examined, and pathologists/consultant biomedical scientists
check the entire slide and not just areas marked as potentially abnormal.
All 10 slides count towards the assessment. Assessment is based on the ability to
distinguish between negative, inadequate, and abnormal slides, the grading of each
abnormal slide, and any instances of missed dyskaryosis.
Pathologists and consultant biomedical scientists may also choose to examine the non-
medical slides for educational purposes, and the EQA facilitator makes sure that these
scores are not included in the performance analysis.
11.4. Trainee staff
Trainee cytology screeners, trainee biomedical scientists and trainee medical staff who
intend to work in the field of cervical cytology are encouraged to participate in EQA.
The scheme is considered to be of educational value for these staff and, although they
are allocated a mark and given the same level of feedback as qualified staff within their
peer group, their results are excluded from any consensus calculation or performance
analysis of the slide set.
11.5. Additional slide sets
If participants choose to view extra slide sets in addition to their allocated EQA slide set,
results from these additional sets are not included in the performance analysis and not
recorded on EQA online.
38
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
12. The assessment
12.1. Recording and submitting responses
All participants in the EQA scheme register their opinion concerning the cytological
pattern on an EQA slide in a format consistent with both standard British Association for
Cytopathology reporting and the standard request form (HMR 101).
Figure 3 equates to the screen viewed by participants in EQA online.
Columns added to the table for special interest slides are not part of the formal
assessment.
The response form is available in paper or electronic format for those participants
unable to access EQA online.
The response form includes mandatory fields for:
personal identity code (only on paper or electronic version, not online)
laboratory code
date of EQA
test set identity
Participants receive instruction on completing whichever format form they use.
Participants must note that:
only one box must be marked in the result column of the cytological pattern
section for each slide
responses must be in the form of a tick, cross or strike (paper/electronic version),
and any other marks are not acceptable and will be recorded as a ‘no response’
a mark (tick, cross or strike) which spans more than one box is recorded as a
‘transcription error’
if a box is marked in error, it must be blocked out completely using ink or
correction fluid (paper version)
failure to mark one box or marking more than one box in any column is recorded
as a ‘transcription error’
Participants are responsible for making sure their responses are entered as intended,
as these cannot be changed after submission. Participants entering their responses
online are prompted to check and confirm their answers before submitting them.
39
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Participants who submit their responses via paper form are made aware of the
penalties for transcription errors. Forms submitted to the EQA facilitator are scored only
as received. There is no opportunity for ‘modifying’ forms prior to marking. It remains
the responsibility of the individual participant to make sure that responses are recorded
correctly.
Participants using EQA online must upload their results immediately after screening
their slides, and while the slides are still in the laboratory.
All participants must keep a paper record of their responses once these have been
submitted until the final results are issued by the scheme. This includes participants
recording their results using EQA online. This is to mitigate against potential loss of
EQA data in the event of IT system failure.
40
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Cytological pattern Result
Result code
HMR101 category Slide number 1 2 3 4 5 6 7 8 9 10
Inadequate (1)
Negative (2)
Borderline change in squamous cells
(8)
Borderline change in endocervical cells
(9)
Low-grade dyskaryosis (3)
High-grade dyskaryosis (moderate) (7)
High-grade dyskaryosis (severe) (4)
invasive squamous carcinoma (5)
Glandular neoplasia of endocervical type
(6)
Glandular neoplasia
(non-cervical)
(0)
Figure 3. Response form
12.2. Following assessment
12.2.1. Release of submitted opinions
Once all participants (or one of the peer groups) within the laboratory have viewed the
EQA slides and recorded and submitted their results, the opinions that have been
provided by the submitting laboratory are released to those participants while the EQA
slides are still in the laboratory.
12.2.2. Interim feedback
EQA online makes interim feedback available to view in the form of submitted opinions.
The scheme makes paper or electronic format interim feedback available for
participants unable to access EQA online. This allows individuals to compare their own
responses with the submitted opinions for the EQA slides at an early point.
Interim feedback is provisional and participants must not assume this will necessarily
match their final result. The interim report form clearly states that the results are
confidential to the individual participant.
Final EQA results are based on consensus opinions that are only available at the end
of the round.
41
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
12.2.3. Final feedback
The scheme provides confidential final feedback to each participant at the end of the
EQA round. This is in a report (or series of reports) and includes an overall score of
performance in the scheme. Verbal feedback on individual performance is not
available.
12.3. Reports
12.3.1. Participant reports
The head of screening quality assurance and the EQA facilitator have joint
authorisation to issue reports to individuals at the end of each completed round.
Participant reports are issued within the 6-month period allocated for each round (no
later than 30 September for the first round, and no later than 31 March for the second
round). Participants must keep their own records of performance in proficiency testing,
including the outcomes of investigations of any unsatisfactory results.
Performance results are a useful record for participants. Limitations are placed on using
these to determine competence. Successful performance may represent evidence of
competency for a particular round but may not reflect ongoing competence. Similarly,
unsatisfactory performance in a particular round may reflect a random departure from a
participant’s normal competence. [1]
12.3.2. Annual report
The annual report includes a summary of participant results and is widely distributed
within the programme and to key stakeholders
12.4. Certificates of participation
All eligible individuals will receive a certificate of participation once the final results
reports have been validated and issued. Eligible laboratories will also receive a
certificate of participation.
12.5. CPD and CME credits
For the purposes of continuing professional development, participants may claim 2
hours CPD/CME credit per completed EQA round.
42
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
13. Slide consensus
13.1. Consensus opinion
Final analysis of participants’ performance is based on consensus agreement. The
correct classification of a slide is based on consensus opinion determined at the end of
the round and is based on an 80% consensus of participant results for each peer
group.
Within the appropriate peer group, which has submitted a valid response for an EQA
slide, at least 80% of the participants will have agreed in their classification of that slide
in order for it to be used to analyse performance. Such calculations only take into
account EQA responses from participants who participate in primary screening and/or
the issuing of reports for the NHSCSP. Responses from participants working in
laboratories that perform only private screening work are excluded from the consensus
calculations.
If a participant fails to provide a response for a slide, or provides multiple responses for
a slide, they are excluded from the consensus calculations for that slide and scored
according to the guidance.
13.2. Data handling and calculations
To simplify data handling and calculations, the EQA facilitator keeps a record of
responses by participant number (not name) for each EQA slide.
Automatic electronic calculation of consensus scoring is acceptable, provided that
these calculations have been previously audited and validated.
Although the calculation of consensus results in a delay in the provision of final
feedback to participants, the difficulties inherent in reaching an agreement on
cytological classification mean that this approach is necessary.
13.3. Slides with consensus different to the submitted classification
If the consensus opinion differs significantly from that of the submitting laboratory and
review panel then the EQA facilitator informs the laboratory and returns the slide
immediately.
This includes any slides where there is a mismatch between the submitted opinion and
the consensus opinion as to whether the slide is negative, inadequate, or abnormal. It
43
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
also applies to slides in the pathologists’ set that are reported with 80% consensus as
high-grade dyskaryosis (moderate) or worse, but have been submitted as showing:
borderline change in squamous cells
borderline change in endocervical cells
low-grade dyskaryosis
The scoring of participants on such slides is based on the consensus opinion.
It is the responsibility of the medical head of the submitting laboratory to take any
necessary action following mismatch between submitted and consensus opinion.
13.4. Consensus on negative, inadequate and normal slides
Consensus agreement on negative, inadequate and abnormal slides is based on all
valid participant responses in the peer group for the particular slide.
13.4.1. Consensus calculations for the distinction between negative, inadequate and
abnormal
To calculate the consensus agreement for the distinction between negative,
inadequate, and abnormal slides, the following codes on the response form are all
deemed abnormal:
3 (low-grade dyskaryosis)
4 (high-grade dyskaryosis (severe))
5 (invasive squamous carcinoma)
6 (glandular neoplasia of endocervical type);
7 (high-grade dyskaryosis (moderate))
8 (borderline change in squamous cells)
9 (borderline change in endocervical cells)
0 (glandular neoplasia (non-cervical)
For each slide, the number of participants who have classified the slide as negative,
inadequate, or abnormal is totalled. If all participants have submitted the same
response then the consensus agreement for the slide is 100%. For slides where there
is more than one classification suggested, the consensus agreement is calculated
using the classification that has the highest number of responses, as follows:
highest number of participants giving the same classification % consensus = x 100
total number of participants giving a valid response for the slide
44
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
13.5. Identifying missed dyskaryosis
Missed dyskaryosis is defined as a participant reporting a slide as negative or
inadequate which is classified by consensus as dyskaryotic.
13.5.1. Consensus for identifying missed dyskaryosis
This is calculated by assessing whether all responses of low-grade dyskaryosis or
worse within the group total 80% or greater.
13.6. Grading consensus for abnormal slides
Consensus agreement on the grading of abnormal slides is based on all valid opinions
reported on the slide by each peer group. The grading consensus for cytology
screeners and checkers is provided for educational feedback only.
The consensus opinion and scoring of pathologists and consultant biomedical scientists
for the grading of abnormal slides is based on EQA responses from pathologists and
consultant biomedical scientists only.
If the consensus agreement (calculated in section 13.4.1) was 80% or greater for an
abnormal classification of a particular slide, then the grading consensus is calculated
for that slide as follows:
highest number of participants giving the same grade % consensus =
x 100 total number of participants giving a valid response for the slide
If 80% consensus is not reached, but the amalgamation of adjacent grades would
enable it to be reached, then this is permitted. The grade with the highest level of
consensus is combined with the grade with the next highest consensus, provided that
the 2 grades in question conform to the following groupings:
borderline change in squamous cells and borderline change in endocervical
cells.
borderline change (squamous and endocervical cells) and low-grade dyskaryosis
low-grade dyskaryosis and high-grade dyskaryosis (moderate)
high-grade dyskaryosis (moderate) and high-grade dyskaryosis (severe)
high-grade dyskaryosis (severe) and invasive squamous carcinoma
high-grade dyskaryosis (severe) and glandular neoplasia of endocervical type,
glandular neoplasia (non-cervical), or both glandular neoplasia categories
invasive squamous carcinoma and either glandular neoplasia of endocervical
type, or
glandular neoplasia (non-cervical), or both glandular neoplasia categories
45
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
glandular neoplasia of endocervical type and glandular neoplasia (non-cervical)
If combining the different categories of borderline change still does not achieve 80%
consensus, amalgamation with a third grade (low-grade dyskaryosis) may be
performed. Similarly, the glandular neoplasia categories may be combined with each
other or another adjacent grade to reach consensus. The amalgamation of both
glandular neoplasia categories plus one other adjacent grade (which has the third
highest level of consensus) is also permitted.
In the unlikely event that the categories on either side of the grade with the highest
level of consensus have the same percentage consensus, follow the guidance given in
Table 1.
These amalgamations exist for scoring purposes only and may not be compatible with
any specific management recommendation.
13.7. Slides not reaching consensus
Slides that have not been reported with 80% consensus as negative, inadequate, or
abnormal are removed from any performance analysis. This means that participants
are scored only against the remaining slides in the set that reach consensus. To
achieve statistical validity, a minimum of 6 slides within any set has to reach
consensus.
13.8. Slides with abnormal consensus but no grade consensus
If a slide has at least 80% consensus for an abnormal classification, but 80%
consensus is not reached for the grading of that slide (even when adjacent grades are
amalgamated), no score is allocated for grading.
Table 1. Amalgamation of grades when both adjacent grades have equal consensus
Grade with highest consensus
Grades with next highest (equal) consensus
Consensus
Low-grade dyskaryosis
Borderline change (squamous and endocervical)
Low-grade dyskaryosis/Borderline change (squamous and endocervical)
High-grade dyskaryosis (moderate)
High-grade dyskaryosis (moderate)
Low-grade dyskaryosis High-grade dyskaryosis (moderate)/High-grade dyskaryosis (severe)
High-grade dyskaryosis (severe)
High-grade dyskaryosis (severe)
High-grade dyskaryosis (moderate) High-grade dyskaryosis (severe)/High-grade dyskaryosis (moderate)
invasive squamous carcinoma
46
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
High-grade dyskaryosis (severe)
invasive squamous carcinoma High-grade dyskaryosis (severe)/ invasive squamous carcinoma Glandular neoplasia of endocervical type
or
?Glandular neoplasia (non-cervical)
High-grade dyskaryosis (severe)
Glandular neoplasia of endocervical type High-grade dyskaryosis (severe)/Glandular neoplasia of endocervical type
Glandular neoplasia (non-cervical)
invasive squamous carcinoma
High-grade dyskaryosis (severe)
invasive squamous carcinoma/High-grade dyskaryosis (severe)
Glandular neoplasia of endocervical type or Glandular neoplasia (non-cervical)
invasive squamous carcinoma
Glandular neoplasia of endocervical type invasive squamous carcinoma/Glandular neoplasia (non-cervical)
Glandular neoplasia (non-cervical)
Glandular neoplasia of endocervical type
High-grade dyskaryosis (severe) Glandular neoplasia of endocervical type/ High-grade dyskaryosis (severe)
invasive squamous carcinoma
Glandular neoplasia (non- cervical)
High-grade dyskaryosis (severe) Glandular neoplasia (non-cervical)/?invasive squamous carcinoma
invasive squamous carcinoma
Glandular neoplasia of endocervical type
High-grade dyskaryosis (severe) or invasive squamous carcinoma Glandular neoplasia of
endocervical type/?Glandular neoplasia (non-cervical)
Glandular neoplasia (non-cervical)
Glandular neoplasia (non- cervical)
High-grade dyskaryosis (severe) or invasive squamous carcinoma Glandular neoplasia of
endocervical type/Glandular neoplasia (non-cervical)
Glandular neoplasia of endocervical type
Glandular neoplasia combined (endocervical and non-cervical)
High-grade dyskaryosis (severe) Glandular neoplasia (endocervical and non- cervical)/?invasive squamous carcinoma
invasive squamous carcinoma
47
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
14. Scoring
14.1. Consensus slides
Participants are only scored on slides in their EQA slide set that reach 80% consensus.
Participants from laboratories where only private screening work is undertaken are
scored against the consensus for each slide calculated using responses from
participants who work in the NHSCSP.
EQA schemes are generally based on 80% consensus, and that consensus may or
may not be correct.
14.2. Cytology screeners and checkers
14.2.1. NIA score
Cytology screeners and checkers are marked on their ability to distinguish between
negative, inadequate, and abnormal slides (an NIA score) as follows:
slide correctly interpreted as negative/inadequate/abnormal = 2 marks
slide incorrectly interpreted = 0 marks
Incorrect interpretations can fall into the following categories:
the slide has a negative consensus and the participant has reported it as
inadequate or abnormal
the slide has an inadequate consensus and the participant has reported it as
negative or abnormal
the slide has an abnormal consensus and the participant has reported it as
negative or inadequate
A scoring matrix for cytology screeners and checkers is given in Appendix 4.
14.2.2. Missed dyskaryosis
Where cytology screeners and checkers have reached consensus that a slide shows
dyskaryosis (80% or more of participants have classified it as low-grade dyskaryosis or
worse), a participant who reports the slide as negative or inadequate is recorded as
having missed dyskaryosis. This counts as substandard performance.
48
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
14.2.3. Educational value
To enhance the educational value of the scheme, qualitative and quantitative feedback
is given to cytology screeners and checkers on their grading of abnormal slides.
14.3. Pathologists and consultant biomedical scientists
14.3.1. NIA score
Pathologists and consultant biomedical scientists are scored on their ability to
distinguish between negative, inadequate, and abnormal slides (an NIA score).
In addition to the NIA score, extra marks are allocated for grading of consensus
abnormal slides:
correct grading of slide = 2 marks
grading of slide within one category either side = 1 mark
grading of slide wider than one category either side = 0 marks
Participants do not lose a mark for reporting a slide that has a consensus report of
borderline change in squamous cells as borderline change in endocervical cells, and
vice versa.
A scoring matrix for pathologists and consultant biomedical scientists is in Appendix 5.
14.3.2. Missed dyskaryosis
If a slide has reached 80% consensus among pathologists and consultant biomedical
scientists as showing dyskaryosis (80% or more of participants have classified it as low-
grade dyskaryosis or worse), a participant who grades the slide as negative or
inadequate is recorded as having missed dyskaryosis. This counts as substandard
performance.
49
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
14.4. Transcription errors
If a participant fails to tick a box on the response form for a slide, or ticks more than one
box for a single slide, this is penalised as an overcall or undercall as follows.
If no box is ticked
No mark is given. If the consensus is that the slide should be reported as dyskaryosis,
this is counted as missed dyskaryosis, and thus as substandard performance.
If 2 boxes are ticked (both abnormal)
If the consensus is abnormal, 2 marks are given for correctly classifying slide as
abnormal (NIA score), then marked for grading using the ticked box that is furthest
away in grade from the consensus grade. For example, if low-grade dyskaryosis and
high-grade dyskaryosis (severe) are both ticked, and the consensus is invasive
squamous carcinoma, the mark is as if the participant has only ticked low-grade
dyskaryosis. If the consensus is negative or inadequate, the participant receives zero
marks.
If negative or abnormal are ticked, and abnormal is ticked
No mark is given. If the consensus is that the slide should be reported as dyskaryosis,
this is counted as missed dyskaryosis, and thus as substandard performance.
If one box is marked inadequate and one is marked negative
No mark is given. If the consensus is that the slide should be reported as dyskaryosis,
this is counted as missed dyskaryosis and thus as substandard performance.
Where a participant has given 2 or more responses for a single slide these responses
are excluded from the consensus calculation for that slide.
14.5. Final scores
In order to make sure that scores are comparable within peer groups (cytology
screeners/checkers or pathologists/consultant biomedical scientists), the total score for
each participant is converted into a percentage of the possible score for the particular
slide set that was used (after any non-consensus slides have been removed).
participant’s score % score =
maximum possible score for the slide set
x 100
50
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
The scheme produces separate lists of scores for cytology screeners and checkers,
and pathologists and consultant biomedical scientists. Scores are listed by participant
code and ranked in percentage score order. Reported scores do not include trainee
staff or staff from laboratories that do not participate in the NHSCSP.
Percentage scores are expressed to 2 decimal places. Values below 4 in the third
decimal place are rounded down. Values of 5 or above are rounded up.
15. Assessment of performance
The formal assessment of personal performance differs for the two peer groups of
participants.
15.1. Cytology screeners and checkers
For cytology screeners and checkers, personal performance is assessed on the
individual’s ability to distinguish between negative, inadequate, and abnormal slides.
Any instance of missed dyskaryosis is also counted.
15.2. Pathologists and consultant biomedical scientists
For pathologists and consultant biomedical scientists, personal performance is
assessed on an individual’s ability to distinguish between negative, inadequate, and
abnormal slides, the grading of cytological patterns for abnormal slides, and any
instance of missed dyskaryosis.
15.3. Identification of substandard performance
Substandard performance is specified as:
a score that falls below the bottom 2.5 percentile point when percentage scores
are placed in rank order for the slide circulation
an instance of missed dyskaryosis
15.3.1. Calculation of the bottom 2.5 percentile point
It is possible to calculate the bottom 2.5 percentile point in Excel by using the inbuilt
function key (fx) on the tool bar. The percentile function can be found in the statistical
function category. The ‘array’ of data is all percentage scores in the rank and the ‘K’
value is the percentile value sought, expressed as a decimal (such as 0.025). The
value obtained is the bottom 2.5 percentile point of all the results. Results falling
51
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
beneath this value are identified as substandard for the purposes of EQA performance
assessment.
Automatic electronic calculation of the bottom 2.5 percentile point is acceptable,
provided that these calculations have been previously audited and validated.
It is also possible to calculate the score at the 2.5 percentile manually, as shown in the
following example:
The calculation is based on an EQA round consisting of 68 percentage scores that range from 100% at
the upper level to 60% at the lower level. The score of the second lowest participant (R2) was 66%; that of the third lowest participant (R3) was 72%.
The formula used is
1 + P(N – 1) = a + b
where a is the integer of the result indicating the ranked position of the score, b is the fraction indicating the proportion of the difference between scores at (a) and (a + 1), P is the percentile (0.025), and N is the
number of participants (68).
1 + 0.025(68 – 1) = a + b
1 + (0.025 x 67) = a + b
1 + 1.675 = a +
b a + b = 2.675
a = 2
b = 0.675
Since a = 2, the score of the second lowest participant is taken as the baseline; if the calculation resulted in a = 1 then the score of the lowest participant would be used as the baseline, or the third lowest participant if the result was a = 3 and so on. With a = 2, the value of b, 0.675, relates to the proportion of the difference between the scores of the second and third lowest participants (if the calculation resulted in a = 1 then the value of b would relate to the difference between the scores of the lowest and second lowest participant and so on).
The proportional difference between the two figures is calculated as follows:
(R3 – R2) x 0.675
(72 – 66) x 0.675
6 x 0.675 = 4.05
The desired percentile is then calculated by adding this difference to the score of the second lowest participant (as a = 2 in the first part of the calculation) as follows:
66 + 4.05 = 70.05%
Any scores below 70.05 are below the 2.5 percentile point and considered to be substandard performance.
Owing to the nature of the scoring system, on some occasions there may be individuals
whose scores equal the 2.5 percentile point with no scores falling below the value. In
52
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
such cases, the scheme does not consider these individuals to have shown
substandard performance.
Participants from private (non-NHS) laboratories or overseas laboratories have their
percentage scores compared to the 2.5 percentile point after calculations are
completed for the other participants. Their individual percentage scores are therefore
not included in the calculations to determine the 2.5 percentile.
Substandard performance is identified by placing the scores for each peer group in rank
order for each circulation. Individuals with scores below the 2.5 percentile point are
classed as having substandard performance for that EQA round.
15.3.2. Missed dyskaryosis
Missed dyskaryosis is identified as a participant report of negative or inadequate for a
slide that has a consensus classification from the relevant peer group of dyskaryosis.
This is reached by combining all valid responses of low-grade dyskaryosis, high-grade
dyskaryosis (moderate), high-grade dyskaryosis (severe), invasive squamous
carcinoma, glandular neoplasia of endocervical type and glandular neoplasia (non-
cervical).
15.3.3. Notification of substandard performance
If a participant scores below the 2.5 percentile and misses a dyskaryosis, this is
deemed to be a single instance of substandard performance for that round.
Any participant with substandard performance receives a confidential letter notifying
them of this, and offering support from the scheme organiser. Although this does not
constitute persistent substandard performance, staff are encouraged to discuss a result
of this nature with their training officer or line manager.
15.4. Persistent substandard performance
Persistent substandard performance is defined as the occurrence of either of the
following in 2 out of 3 consecutive EQA rounds:
scoring below the 2.5 percentile point
missed dyskaryosis
After each round, results are analysed to identify persistent substandard performance.
This is based on the participant’s EQA results for the three most recent rounds. The
EQA facilitator keeps a record of participants’ previous scores complete with comments
so that persistent substandard performance can be easily identified.
53
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
A participant is not classified as showing persistent substandard performance on the
basis of the results of a single round. For example, if a participant misses dyskaryosis
on 2 or more occasions in one round, or misses dyskaryosis on one or more occasions
and also falls below the 2.5 percentile point in that round, his or her performance is
graded as substandard for that round, but not as persistent substandard performance.
15.5. Action points for persistent substandard performance
Action points, and the remedial measures taken when they are reached, are defined by
the NQAAP (histopathology and cytopathology).
15.5.1. First action point
The first action point is reached when persistent substandard performance is first
identified.
15.5.2. Second action point
The second action point is reached when a participant’s performance continues to be
substandard in 2 out of 3 consecutive rounds after the first action point has been
reached. Persistent substandard performance is determined on a ‘rolling’ basis (using
the results of the last 3 consecutive rounds). Substandard performance in either of the
next 2 rounds after triggering the first action point means that the participant has
reached the second action point.
A table, which demonstrates the various combinations of results that can lead to the
first and second action points, depending on an individual’s pattern of persistent
substandard performance, is in Appendix 6.
The specific action to be taken when action points are reached is different for each peer
group. The pathways for each action point are in Appendices 7.1 and 7.2.
15.5.3. Developing a remedial action plan
Participants identified as persistent substandard performers are required to produce
and implement a remedial action plan. The participant (jointly with the laboratory
manager when necessary) will submit a declaration to the scheme organiser to confirm
and evidence that the plan has been implemented and appropriate action taken. The
pathway is outlined in Appendix 7.1.
54
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
The minimum requirements for a remedial action plan are:
explanation of the reason for substandard performance
review of workload figures for cervical cytology
review of screener and checker performance data for the period of EQA
substandard performance consisting of abnormal sensitivities, high grade
sensitivities and reporting profiles
review of pathologist and consultant biomedical scientist performance data for
the period of EQA substandard performance consisting of positive predictive
value (PPV), abnormal predictive value (APV) and Referral Value (RV)
proposals for a review of in-house teaching sets or the scheduled attendance of
suitable formal training
reflective statement taking into account the wider perspective of the working
environment and current performance
15.5.4. Participants from outside the UK
Non-UK laboratories declare their professional lines of accountability within their
employing organisation and regulatory body under the terms and conditions of the EQA
agreement. The EQA scheme reports persistent poor performance to the relevant
regulatory body.
55
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
16. Communication to and from the EQA
scheme
16.1. Constructive criticism, comments and compliments
The EQA scheme for gynaecological cytopathology considers any concern or negative
comment relating to any aspect of its operation or management to be a complaint.
Scheme participants are welcome to express concerns or suggest improvements via
email, telephone or letter. The EQA service actively seeks feedback (both positive and
negative) through a national annual survey and a participants’ forum.
Verbal and written communications are managed locally. The scheme defines the term
‘complaint’ as dissatisfaction about or criticism of the EQA operation or management. A
complaint does not have to be labelled as such by the sender for it to be treated as a
complaint by the scheme if it displays the criteria indicated. All communications are
logged and acted on as needed, and used to help identify system improvements.
16.2. Formal complaints
Formal complaints go directly to the EQA facilitator in the first instance. In the event of
failure to resolve a complaint, the individual in question can complain directly to the
scheme organiser. If the issue is still unresolved, the participant can then refer his or
her complaint to the chair of NQAAP. The NQAAP decision is final.
The complaints procedure (Appendix 8) describes how participants can lodge a formal
complaint regarding any aspect of scheme operation or management.
16.3. Formal appeals
If EQA participants wish to formally appeal regarding their performance assessment or
results, they must contact the EQA facilitator in the first instance. In the event of
escalation, the individual in question can complain directly to the scheme organiser.
Ultimately, the participant can refer their appeal to the chair of NQAAP. The NQAAP
decision is final.
The appeals procedure (Appendix 9) describes how participants can lodge a formal
appeal against an evaluation of their EQA performance.
56
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
16.4. Protocol changes
The protocol may undergo changes when improvement opportunities are identified, or
when non-conformity within a process has been identified and requires corrective
and/or preventive action.
The changes are listed in the revision history at the beginning of the protocol. The
revised protocol will be uploaded to EQA online, and a PDF version provided to
laboratories that submit their responses manually. All participants are required to
confirm, via EQA online or through the submission of a written declaration, they have
read and accept the terms specified in the protocol.
Changes may also arise as a consequence of participant feedback or direct request
from the operational group, steering group or management board.
The scheme uses a change control procedure to prioritise and manage all changes.
The EQA facilitator will notify participants of agreed changes to the scheme and the
date when they are to come into effect.
16.5. Participants’ meetings
Participants’ meetings/workshops provide a forum for staff to view scheme results
comparisons, discuss operational aspects of the scheme and review any challenging
slides and/or educational cases. The feedback from the participants’ annual meeting is
managed through the EQA operational group. A summary is provided in the annual
report.
The meetings are held annually and open to all staff. Laboratories are encouraged to
send at least one member of staff who can then cascade information to colleagues.
16.6. Audit
The EQA operational group is responsible for setting the scope of the audit strategy
and preparing the internal audit programme. The EQA management board is
responsible for authorising the audit strategy and programme and may advise on
adjustments if considered necessary.
The scheme organiser is responsible for notifying participants if audit findings indicate
that non-conformities may have affected participant EQA results.
The head of screening quality assurance is responsible for all UKAS related
communications, and for making sure the overall requirements of external assessments
are met.
57
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
16.7. Surveys
Short surveys are used periodically to obtain feedback or views from participants on
actual or proposed scheme activity.
There is an annual survey, which all participants are encouraged to complete, either to
register their satisfaction with the EQA service or use as an opportunity to provide
constructive criticism, comments and/or suggestions for improvement.
Findings from the participant surveys are managed through the EQA operational group.
A summary is included in the annual report.
58
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
16.8. Annual report
The scheme produces an annual report for:
participants
members of the EQA management board, steering group and operational group
PHE director of screening
PHE head of YPA screening programmes
PHE national screening QA service lead
PHE regional cervical screening QA portfolio leads (for cascade to regional
cervical screening QA staff)
National Quality Assurance Advisory Panel (NQAAP) for Cellular Pathology
59
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
References
1. ISO 17043:2010 Conformity assessment – General requirements for proficiency
testing.
2. ISO 15189:2012 Medical Laboratories – Requirements for quality and
competence.
3. Achievable Standards, Benchmarks for Reporting and Criteria for Evaluating
Cervical Cytopathology, 3rd edition. NHS Cancer Screening Programmes, 2013
(NHSCSP Publication No 1).
4. Recommended Code of Practice for Laboratories Participating in the UK Cervical
Screening Programmes. London, British Association for Cytopathology, 2015.
5. ‘The retention and storage of pathological records and specimens’, fifth edition
April 2015
60
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
Appendix 1: procedure for reporting and
investigating alleged collusion
1. Purpose
To provide a procedure for reporting and investigating collusion or any behaviour
witnessed or strongly suspected, whether deliberate or not, which could compromise
the validity of EQA results and consequently damage the integrity of the scheme.
2. Scope
This procedure applies to all scheme participants and EQA staff.
3. Policy
Participants must not pass on information during the EQA round to allow his or her
opinions to be submitted by another. The scheme defines such behaviour as collusion.
Collusion or any behaviour that threatens the integrity of the scheme is taken seriously
and will be escalated to the highest authorities.
4. Responsibilities and authorities
All participants and EQA staff have a responsibility for reporting collusion or any
behaviour, which could potentially undermine the scheme’s integrity.
The scheme organiser is responsible for responding to a reported allegation of
collusion. The scheme organiser is responsible for collaborating with the individuals
involved and the appropriate laboratory lead.
The appropriate laboratory leads are responsible for any local investigation and
notifying the outcome to the scheme organiser.
If collusion is proven, the appropriate laboratory lead is obligated to refer the matter to
the relevant professional regulatory body and notify the scheme organiser.
5. Procedure
In the first instance, the EQA facilitator must be notified in writing if collusion is
witnessed or strongly suspected. The EQA facilitator refers the information to the
scheme organiser.
61
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
The procedure for managing allegations of collusion is set out below.
5.1 Medical staff
The scheme organiser writes to individuals separately via the EQA facilitator within 7
days of being notified about the allegation.
The recipients are asked to acknowledge receipt of the letter and respond to the
scheme organiser within 14 days. The recipients are advised to identify themselves to
the clinical head of the laboratory and offer an explanation for the allegation of
collusion.
The scheme organiser writes to the clinical head of the laboratory setting out the
concern, and explains that the individuals are required to identify themselves to him or
her. The email does not identify the individuals concerned. The clinical head of the
laboratory is asked to acknowledge the letter and respond to the scheme organiser
within 14 days of receipt.
The clinical head of the laboratory provides appropriate feedback and guidance to the
individuals concerned. The clinical head of the laboratory writes to the scheme
organiser to confirm the outcome of any local investigation within a further 14 days.
If no acknowledgement is forthcoming by 21 days then a reminder is sent. If no reply is
received after a further 7 days, the scheme organiser notifies the chair of NQAAP. The
NQAAP chair speaks with the head of screening QA about further action.
If collusion is proven, the EQA round of the individuals concerned is invalidated and
counted as sub-standard performance. The clinical laboratory lead reports the matter to
the organisation’s HR department, SQAS and relevant regulatory body. The clinical
laboratory lead notifies the scheme organiser who in turn notifies NQAAP.
5.2 Non-medical staff
The scheme organiser writes to individuals separately via the EQA facilitator within 7
days of being notified about the allegation.
The recipients are asked to acknowledge the letter and respond to the scheme
organiser within 14 days of receipt. The recipients are advised to identify themselves to
the scientific head of the laboratory and offer an explanation for the allegation of
collusion.
The scheme organiser writes to the laboratory scientific head for cervical cytology
setting out the concern, and confirms that the individuals have been advised to identify
themselves to him or her. The email does not identify the individuals concerned.
62
Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology
The scientific head of laboratory is asked to acknowledge the letter and respond to the
scheme organiser within 14 days of receipt.
The scientific head of laboratory provides appropriate feedback and guidance to the
individuals concerned. The scientific head of the laboratory writes to the scheme
organiser to confirm the outcome of any local investigation within a further 14 days.
The laboratory’s scientific head informs the lead consultant for the cytology service and
the medical head of department. If it is the scientific head that is under investigation,
then the copy letter goes to the consultant pathologist responsible for reporting cervical
cytology.
If no acknowledgement is forthcoming by 21 days then a reminder is sent. If there is still
no response after a further 7 days, then the scheme organiser notifies the NQAAP
chair. The NQAAP chair speaks with the head of screening QA about further action.
If collusion is proven, the EQA round of the individuals concerned is invalidated and
counted as sub-standard performance. The scientific laboratory lead reports the matter
to the organisation’s HR department, SQAS and the relevant regulatory body. The
scientific laboratory lead notifies the scheme organiser who in turn notifies NQAAP.
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
63
Slides cleaned and delivered to next
laboratory
Identify
participants
and prepare
forms
Slide
circulation
begins
Slides/forms delivered to laboratory
Submitted classifications released once EQA complete
Interim feedback given
Appendix 2: EQA timeline
Contact laboratories to request slides for inclusion in the rounds
Slide
review
Assembly of
slide sets
Set dates for
participation
Late
participation
Statistical analysis and identification of persistent substandard performance
Final
feedback
issued
Slide
circulation
ends
Slides returned.
Participants’ meeting held
annually following
completion of two EQA rounds
64
Appendix 3: referral sheet
Referral of abnormal slides for reporting
Referral for reporting is on the basis that the slides indicated are, in your opinion,
potentially or probably abnormal. You will not be marked on the basis of your responses
on this form. Please treat the slides as you would in normal practice within your
laboratory (for example, ringing, dotting or marking as appropriate).
Cytology screener
If you are a cytology screener, please indicate with an ‘R’ the slides you would refer to a
checker or pathologist for reporting on the basis of your opinion. The pathologist
examines the slide according to his or her normal practice and forms his or her own
opinion.
Checker
If you are a checker, please indicate with an ‘A’ the slides you would refer to a
pathologist for reporting on the basis of your opinion. Please look only at slides marked
‘R’ by the cytology screener. The pathologist examines the slide according to his or her
normal practice and forms his or her own opinion.
Hospital ....................................
Slide set identification ..............
Please mark in the appropriate box
Slide number
1
2
3
4
5
6
7
8
9
10
Cytology screener
Checker
Note that this form is not part of the assessment and is retained for laboratory records
only.
65
Appendix 4: scoring matrix for cytology
screeners/checkers
Consensus
Negative Inadequate Abnormal
Negative 2 0 0
Inadequate 0 2 0
Borderline changes 0 0 2
Low-grade dyskaryosis 0 0 2
High-grade dyskaryosis (moderate) 0 0 2
High-grade dyskaryosis (severe) 0 0 2
invasive squamous carcinoma 0 0 2
Glandular neoplasia of endocervical type 0 0 2
Glandular neoplasia (non-cervical) 0 0 2
Slide correctly interpreted Slide incorrectly interpreted
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
66
Appendix 5: scoring matrix for pathologists/
consultant biomedical scientists
Response
Inadequate
Negative
Borderline Low- grade
High- grade Moderate
High- grade Severe
Invasive GN endocx
GN non-cx
Co
ns
en
su
s
Inadequate 2 0 0 0 0 0 0 0 0
Negative 0 2 0 0 0 0 0 0 0
Abnormal/ no grade
0 0 2 2 2 2 2 2 2
Borderline 0 0 4 3 2 2 2 2 2
Bord/Low- grade
0 0 4 4 3 2 2 2 2
Low-grade 0 0 3 4 3 2 2 2 2
Low- grade/ HG Mod
0
0
3
4
4
3
2
2
2
High-grade Moderate
0 0 2 3 4 3 2 2 2
HG Mod/ HG Sev
0 0 2 3 4 4 3 3 3
High-grade Severe
0 0 2 2 3 4 3 3 3
HG Sev/ Inv
0 0 2 2 3 4 4 3 3
HG Sev/ GN endocx
0
0
2
2
3
4
3
4
3
HG Sev/ GN non- cx
0
0
2
2
3
4
3
3
4
HG Sev/ GN
0 0 2 2 3 4 3 4 4
Invasive 0 0 2 2 2 3 4 3 3
Inv/ ?GN endocx
0
0
2
2
2
3
4
4
3
Inv/ GN non-cx
0
0
2
2
2
3
4
3
4
Inv/ GN 0 0 2 2 2 3 4 4 4
GN endocx
0 0 2 2 2 3 3 4 3
GN non- cx
0 0 2 2 2 3 3 3 4
GN (GN endocx/ GN non- cx)
0
0
2
2
2
3
3
4
4
Slide correctly interpreted Slide incorrectly interpreted
One grade from consensus opinion > one grade from consensus opinion
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
67
Appendix 6: table of performance examples with action points
1st Round 2nd Round 3rd Round 4th Round 5th Round 6th Round 7th Round
Rounds contributing to 1st Action Point
Rounds contributing to 2nd Action Point
Participant A Substandard Satisfactory Substandard 1st Action Point
Substandard 2nd Action Point
1, 2 & 3 2, 3 & 4
Participant B Substandard Satisfactory Substandard 1st Action Point
Satisfactory Substandard 2nd Action Point
1, 2 & 3 3, 4 & 5
Participant C Substandard Non-valid non-participation 1st Action Point
Satisfactory Substandard 2nd Action Point
1 & 2 2, 3 & 4
Participant D Substandard Substandard 1st Action Point
Satisfactory Satisfactory Substandard Satisfactory Substandard 1st Action Point
1 & 2 5, 6 & 7
-
Participant E Substandard Substandard 1st Action Point
Substandard 2nd Action Point
1 & 2 1, 2 & 3
Participant F Substandard Substandard 1st Action Point
Satisfactory Substandard 2nd Action Point
1 & 2 2, 3 & 4
Participant G Substandard Substandard 1st Action Point
Satisfactory Satisfactory Substandard Substandard 1st Action Point
1 & 2 4, 5 & 6
-
Participant H Substandard Satisfactory Substandard 1st Action Point
Non-valid non-participation 2nd Action Point
1, 2 & 3 2, 3 & 4
Participant I Substandard Substandard 1st Action Point
Satisfactory Non-valid non-participation 2nd Action Point
1 & 2 2, 3 & 4
Participant J Non-valid non-participation
Substandard 1st Action Point
Satisfactory Satisfactory Non-valid non-participation
Satisfactory Satisfactory 1 & 2 -
Participant K Satisfactory Non-valid non-participation
Satisfactory
Non-valid non-participation Cessation of screening
- -
Participant L Substandard Satisfactory Non-valid Non-participation 1st Action Point
Satisfactory
Non-valid Non-pariticpation Cessation of screening
1, 2 & 3 -
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
68
Appendix 7.1: flowchart of actions taken
when first action point is reached
EQA facilitator informs scheme
organiser
Scheme organiser sends
reminder
If no reply within 3 weeks
NQAAP chair informed
First action point is
reached
Medical staff/Consultant
biomedical scientists Non-medical staff
EQA facilitator informs scheme
organiser
Scheme organiser writes to
participant/s
Participant confirms and
evidences implementation of
action plan to scheme
organiser within 4 weeks
Scheme organiser writes to
participant/s (cc letter to
scientific head and medical
head of department)
Scientific head confirms and
evidences implementation of
action plan to scheme
organiser within 4 weeks
Implementation of action plan
not confirmed
Scheme organiser sends
reminder
If no reply within 3 weeks
NQAAP chair informed
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
69
Appendix 7.2: flowchart of actions taken
when second action point is reached
Chair corresponds with
individual via EQA facilitator
If lack of co-operation or no
reasonable explanation
Chair of Joint Working Group on
QA informed
Professional performance
committee of RCPath/Trust
medical director/Health & Care
Professions Council informed
Professional performance
committee review panel
convened
Second action point is
reached
Medical staff/Consultant
biomedical scientists Non-medical staff
Scheme organiser informs
NQAPP chair
Investigation panel convened
Panel seeks possible
explanations
Scheme organiser informs
NQAPP chair and writes to
individual/s (copy scientific
head and medical head of
department)
NQAPP chair and head of
screening QA agree further
action
To
be
co
mp
lete
d w
ith
in 4
wee
ks
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
70
Appendix 8: formal complaints procedure
1. Purpose
This guidance provides a procedure for dealing with complaints from all stakeholders in
the event of dissatisfaction with any aspect of scheme operation or management.
2. Scope
This procedure applies to all participants, EQA staff and stakeholders.
3. Responsibilities and authorities
Participants are responsible for following the procedure if making a complaint. EQA
staff are responsible for taking appropriate action and offering a solution within the
required timeframes.
4. Procedure
If a participant wishes to make a formal complaint, then he or she must do so by writing
to the EQA facilitator.
The EQA facilitator sends a holding response within three working days of receiving the
complaint.
The complaint is logged together with a summary of the communication and
subsequent discussions that may involve other EQA staff including the scheme
organiser.
The complainant can expect to receive a response offering a solution within 28 working
days.
If the matter is not resolved to the satisfaction of the complainant then the EQA
facilitator will refer the matter to the scheme organiser. The scheme organiser will
provide a response within 28 working days.
If the matter is still not resolved then the scheme organiser will escalate it to the EQA
management board. The EQA management board decision is final. A final response
can be expected within 28 working days.
If the complainant remains dissatisfied with the outcome, or the matter becomes
intractable, then they can self-refer the matter directly to NQAAP.
National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol
71
Appendix 9: appeals procedure
1. Purpose
This guidance provides a procedure for dealing with participants in the event of an
appeal against an evaluation of their EQA performance.
2. Scope
This procedure applies to all participants.
3. Responsibilities and authorities
Participants are responsible for making a timely appeal against an evaluation of their
EQA performance. EQA staff are responsible for taking appropriate action and offering
a solution within the required timeframes.
4. Procedure
If a participant wishes to appeal against an evaluation of their EQA performance, then
he or she may do so by writing to the EQA facilitator within 28 working days upon
receipt of their final results report.
The EQA facilitator sends a holding response within three working days of receiving the
appeal.
The appeal is logged together with a summary of the communication and subsequent
discussions that may involve other EQA staff including the scheme organiser.
The appellant can expect to receive a response offering a solution within 28 working
days.
If the matter is not resolved to the satisfaction of the appellant then the matter is
referred to the scheme organiser. A response is provided within 28 working days.
If the matter is still not resolved then it will be escalated to the EQA management
board. The EQA management board decision is final. A final response can be expected
within 28 working days.
If the appellant remains dissatisfied with the outcome, or the matter becomes intractable
then they can self-refer the matter directly to NQAAP.