1 workforce assurance tool training guidance pack - draft nhs north west friday, 11 january 2013
TRANSCRIPT
1
Workforce Assurance ToolTraining Guidance Pack - DRAFTNHS North WestFriday, 11 January 2013
2
Contents• The need for effective workforce assurance
• Assurance framework overview
• Current assurance challenges and how the system will meet these challenges
• The intended benefits of the system
• Principles underpinning the system
• Key components of the system
• How the system can by used by different groups
• Expected levels of efforts and return to use the system
• The data that the system will have access to
• The main components of the tool
• System views, navigation and functionality
• Triggers, metrics and data thresholds
• Access to the system
• User exercises
• Key Questions and Answers
Module 1
Background and Context
Module 2
The system and how it will work
Module 3
Intro to the Tool
Module 4
FAQ’s
3
Module 1Background and Context
4
The need for effective workforce assurancea. Right people, right place, right time
An appropriately sized, skilled, organised and motivated workforce is critical to delivering efficient, quality & safe services.
b. Managing RiskWorkforce assurance in the NHS is the process of managing risk and assuring that the composition of the workforce can deliver cost effective, safe and high quality care. Regardless of reforms to the NHS that result from ‘Liberating the NHS’, the need for assurance processes that review and challenge workforce plans is likely to remain.
c. Quality & SafetyWho leads this process in the new system architecture is yet to be determined but it is critical providers are given the tools to review the impact of workforce changes on quality and safety for themselves – this is an important part of the SHAs’ legacy.
d. Manage ChangeAssurance is needed due to the pressures and inherent risks associated with large scale workforce change from:
• Tough financial settlements – workforce expenditures represent 70% of the total NHS spend and although in recent years workforce numbers have increased inline with overall spending growth, there is now a very significant productivity challenge related to achieving the overall £20 billion national QIPP target.
• Increasing demands for services, and the challenges and focus resulting from the Robert Francis Inquiry into Mid-Staffordshire NHS Foundation Trust.
3 – Tool1 – Background 2 – System 4 – FAQ’s
5
The need for effective workforce assurance
Workforce assurance has impacts at all levels of the system:
• on providers to improve their workforce planning, monitoring systems and impact analysis of their increased productivity efforts. Fundamentally assessing their workforce changes and the impact on quality and safety
• on education commissioners to make better education commissioning decisions
• on service commissioners to undertake assurance with respect to quality and safety of provider delivery plans
• on SHAs with assurance responsibilities to exercise appropriate judgement given the system pressures.
3 – Tool1 – Background 2 – System 4 – FAQ’s
6
The Workforce Assurance Framework provides a method for achieving system assurance that the planned and available workforce can deliver safe and quality services.
The assurance framework
1 – Key lines of enquiry referenced in DH KLOE on Safety and Quality Assurance of Workforce Changes
• The Framework provides a means to identify both current workforce issues and provides an early warning system when reviewing projected workforce changes.
• The assurance framework also reflects the key lines of enquiry agreed by the DH Operations Board1
• The Department of Health are currently developing a national framework for workforce assurance.
• This document describes an approach that will support NHS organisations in implementing this framework in a consistent, transparent and highly automated manner.
NB. Whilst the framework is, to a limited degree, implemented manually there are significant advantages in terms of automation and empowerment for Trusts to embedding it within a tool.
3 – Tool1 – Background 2 – System 4 – FAQ’s
7
The need for a robust system to support workforce assurance
• Much of the data collected is currently retrospective and not forward looking
• Process, dashboards and metrics collected are different in different regions
• Individual indicators are often looked at in silos
• Current processes are not supported by robust statistical analysis
• Current processes are labour intensive as mainly manual processes
• Currently workforce plans go backwards and forwards between Trusts and Assurers, increasing timescales / tensions
Current Challenges
• The tool will use trend data to project how the picture might change over time which can then be compared to actual performance
• Good practice adopted consistently across Trusts and regions with a collation of all existing metrics
• Enables consideration of the importance of interrelationship between indicators and triangulation of workforce, activity and finance
• Assurance processes underpinned by detailed information base and statistical analysis including detailed literature review
• Automation reduced manual data manipulation and handling burden
• Greater visibility of assurance expectations between Trusts and Assurers – a shared tool that can be also be used by individual Providers
Solutions from the tool
Expertise from 8 of 10 SHAs and the Department of Health have been drawn upon to support system design and to address the current limitations associated with attempting workforce assurance using static data and manual processes. The key challenges identified by this group in relation to workforce assurance and the related desired benefits of the system are detailed below:
3 – Tool1 – Background 2 – System 4 – FAQ’s
8
Module 2The system and how it will work
9
Better use of technology will help achieve thisDuring the proof of concept stage, a series of challenges that were most appropriately resolved through the use of technology were:
•The need to collate and manage huge amounts of both structured and unstructured data – this is currently extremely time consuming both for Trusts and SHAs;
•The need to display dynamic / regionalised views of this data in a consistent way;
•The need to compare and contrast on an equal footing across all areas of the system;
•The need to reduce the level of bureaucracy involved in metric analysis and performance management;
•The need to retain and access a clear audit trail for all analysis and action;
•The need to enable the assurance process and outputs to be “agile” enough to respond to changes in the operating environment;
•The need to support workflow and collaboration (again, with a clear audit trail); and
•The need to test robust workforce scenarios
3 – Tool1 – Background 2 – System 4 – FAQ’s
10
A different approach to assurance
What is different?
• A systematic approach enabling rigorous, structured evaluation
• Forward and historic looking analysis
• A statistically robust underpinning for assurance analysis (correlation analysis will help set metric thresholds and identify new areas in need of monitoring)
• Analysis of correlations and relationships between metrics
• Escalation routines to enable consideration of escalation in structured way
This enables differing groups of users to:
• Provide a tool and legacy for trusts so that they can self-assure and pre-emptively respond to concerns of assurers
• Call on robust information and analysis as support for key decision making
• Take a more integrated approach to assurance activities (looking across activity, finance, quality and workforce)
• Forecast, and indentify correlations within, data to suggest expected trajectory and to see how seemingly unconnected triggers interact within an organisation
• Create and drive top table discussions
• Change the nature of assurance engagement with Trust Boards and Management Teams
• Justify decisions
3 – Tool1 – Background 2 – System 4 – FAQ’s
11
What the tool is intended for...
The Tool IS intended to provide:• The workforce element that will link with
wider system level assurance• Triangulation between workforce,
activity and finance data• A national platform to workforce
assurance that can be adapted as appropriate for each SHA
• Best practice approach that is built on existing published literature
• An approach that recognises that assurance criteria will change and be refined over time
The Tool IS NOT intended to be:• A tool to publish information• A performance management tool• A data collection tool; existing data is
used• A directorate, divisional and professional
assurance tool as other resources are available for this
3 – Tool1 – Background 2 – System 4 – FAQ’s
12
Underlying principles of the tool1. Effective workforce assurance is a critical component of helping ensure the ongoing efficient
and effective delivery of services across the NHS system.
2. Organisations should be supported to be rigorous in their self assurance processes, to improve outcomes and reduce the need for external assurance.
3. By using data appropriately, users can be supported to make better assurance decisions. This is a decision support tool, not a decision making tool.
4. Consistent high levels of assurance across regions.
5. Considers a cross section of demand, activity, quality, finance and service information rather than considering workforce alone.
6. Balances forward looking assessments with previous performance: i.e. Historic dashboards are a useful input but do not tell the whole story.
7. To minimise effort, Nationally available data will be used wherever possible to reduce the administrative burden.
8. Differences in data quality and availability need to be allowed for and highlighted within the system.
9. Throughout the life of the tool, the system will be actively managed so that as new national datasets become available each period they are uploaded into the tool.
Fundamental Tenets
WorkforceAssurance in
Practice
Data usage
3 – Tool1 – Background 2 – System 4 – FAQ’s
13
How it will be used by different audiencesProvider Commissioner System Assurer
KeyFocus
Supporting trusts to be rigorous in their self assurance processes and to
benchmark
Supporting commissioners to discharge their assurance responsibilities
Providing a framework for high quality, data driven workforce assurance and
education commissioning
Common Themes
• Consistency and transparency of assurance approach • More effective use of available data• Reduction in manual processes and interchange of information
What you might do with the
system and how you
might use it
• Service leads and clinicians work in collaboration with workforce planners to identify potential future workforce risks, as well as pre-empting and mitigating assurance concerns
• Clinicians and service leads use to help quantify the links between workforce, safety and quality and benchmark against peers
• Potential for Executive teams and Board member use to provide key workforce risk assessment
• Service commissioners use to assess and consider proposed workforce plans using structured, data driven, good practice approach
• Support discharge of workforce assurance responsibilities
• Support workforce planning impact assessment
• Support assessment of system wide trends and failure points
System not designed for
• Detailed operational management at ward and department level
• Performance management of providers
• Performance management of the system
Advantages
• Trust can “see what the assurer sees”
• Trust can validate plans BEFORE submission
• Trust can recognise and pre-empt assurance concerns at an organisational level
• Greater insight into the strengths of individual providers
• Communication mechanism with provider around workforce strength
• Standardised and consistent view of Trusts in designated peer groups
• Interactive workflow to achieve assurance with full audit trail
• Trusts should be better prepared up front
3 – Tool1 – Background 2 – System 4 – FAQ’s
14
Scenario 2: In-year reviewScenario 1: Workforce Planning
Provider worked example
Consults the dashboard to view overall assurance level
• A Trust works with clinicians, service managers and finance to develop a workforce plan.
• Before finalisation, Trust uses the tool to review the plan. This enables the Trust to pinpoint areas of risk and highlight where there are tensions between activity, finance and workforce plans.
• The Trust uses the comparative information, threshold and trend information from the tool to engage in a data driven conversation with clinicians and service managers as to whether alternative approaches need to be considered.
• In some areas the Trust concludes that despite the indication of risk their plans are in fact sound. In other areas the Trust decides to enhance their plans.
• The Trust submits their amended plans with a data driven narrative highlighting why they believe they are sound in perceived risk areas.
• The assurer receives the plans and can consider in light of the Trust’s considered risk assessment.
• Trend projections on dashboard highlight that deviations from plan for one team within the Trust could be indicative of potential quality and patient safety risks if left unchecked.
• Trust uses projection information to review potential risks and inform internal risk prioritisation processes.
• If the Trust deems that action is required, they use this information to support engagement between the team’s clinicians and service managers as to nature and extent of concerns.
• Appropriate action taken within that team, including development of mitigation plans and reworking forecasts as required.
Organisations can also use this tool to review current workforce considerations and should always use the tool in conjunction with more qualitative intelligence
3 – Tool1 – Background 2 – System 4 – FAQ’s
15
Commissioner / assurer worked exampleScenario 1: Commissioner demand management
Consults the dashboard to view overall assurance level
• Commissioning team decide to test the effect on workforce in Trust A from their demand management QIPP plans.
• Commissioners use the tool to model the activity and income changes on Trust A and understand any workforce, quality or safety issues this may cause.
• The Commissioner chooses a scenario which gives the least risk, as seen by the relative score from the dashboard, and discusses impacts with Trust A.
• Trust A uses the same scenario and discusses the dashboard score with managers and clinicians to understand the deliverability of the QIPP plan.
• The Commissioner and Trust A agree the scenario for demand management and the potential effect on the Trust and target decommissioning and resource toward delivering the target together.
Scenario 2: Assurance during transition*
• Assurer chooses the organisations to be assured within the tool and opens dashboards for each of these organisations in a separate web browser window
• Assurer reviews comparability of potential risks across the group, including assessing trends and changing performance over time.
• Assurer prepares guidance on key comparable risks displayed by the dashboard and distributes to providers while also offering support and resource to help mitigate potential risks.
• The assurer continues to monitor month by month performance against the key risk areas and works with providers to target mitigation.
* Assurance will not just be limited to transition but transition is used as an example due to the heightened risk environment
3 – Tool1 – Background 2 – System 4 – FAQ’s
16
Data included in the initial build
• Activity: Health Episode Statistics, Quality Outcomes Framework, Unify, forward looking activity plans, Information Centre
• Workforce: ESR, Census, forward looking workforce plans• Quality/Safety: STEIS, Unify, NHS Patient and Staff Survey, IC Complaints data base,
NPSA Incident and CAS systems, HPA, Monitor • Finance: FIMS, forward looking finance plans, Monitor
Sector specific data that would be useful in the
initial build but are not available
• The acute sector and national datasets provide the richest streams of data for assurance as this has been the focus of reporting and data collection for the NHS / DH in recent years
• Mental Health and Community services will be partially covered due to the absence of quality national data that will limit the robustness of framework (non workforce information primarily)
• Primary care data is not robust enough to provide for primary care workforce assurance and therefore primary care data will be looked at in relation to Provider Assurance and give a perspective on the local health economy
Sector specific data that will not be included
• The following sectors will not be covered in the system and therefore related data will note be included:
• Dental practices• Ophthalmology• Pharmacy• Private sector• Social Care
Data within the system3 – Tool1 – Background 2 – System 4 – FAQ’s
17
Input required from Trusts
Levelof Input
BenefitsReceived
The tool has been designed to require zero / minimal input from Trusts, and can operate with or without forward workforce plans. However, with these plans the value delivered from the tool with be far greater.
Adding workforce plans, with summary finance and activity information
Without Plans:
the tool will provide triangulation, comparisons, and projection using national data sets covering workforce, quality, safety, activity and finance.
With Plans:
This will include comparison against plan functionality and the ability to use trend projections to test the quality and safety implications of the plan.
The scenario modelling element of the tool can be done without providing visibility of the plan to assurers.
Do Trusts have to input data to the tool?
All data will be provided centrally with two potential exceptions:•If Trusts wish to upload a workforce plan / scenario into a private part of the tool for review (i.e. Before I submit my plan, can I check if it will flag up any major areas of concern at the SHA?), then this will be possible at the users discretion; and•Where a regional collection process does not exist, then approved plans may be uploaded by Trusts at their own discretion.
3 – Tool1 – Background 2 – System 4 – FAQ’s
18
Module 3: part 1Introduction to the Tool
19
My Analysis My ResourcesMy Actions
OrganisationDashboard
MetricDashboard
Help
Workforce Assurance Portal
My Flags
Key Components
My QualitativeIndicators
My Plans
My Threads
My Notes
My Assurance Logs
Supporting Materials
Community
Data Sources
FAQ
Workforce Dictionary
3 – Tool1 – Background 2 – System 4 – FAQ’s
20 20
Purpose:• This component contains the primary dashboards within
the system that operate at regional, organisation and specific metric levels.
• Each dashboard highlights areas of potential workforce risk and provides functionality to allow users to drill into risk areas to see more detail.
• The dashboards also allow users to view the underlying data, and perform comparisons against peer organisations.
Key features:• Drill down functionality to enable users to see more detail• Further information can be found on each metric, including
how the metric has been calculated.• The source data indicators can also be viewed.• Links to enable users to jump straight to actions from the
dashboards.• Geographic mapping at a regional level
My analysis Regional User View
Organisational Dashboard
Metric Dashboard
3 – Tool1 – Background 2 – System 4 – FAQ’s
21 21
Clicking on the assurance cheese opens
an Organisation summary panel
The organisation summary panel shows a
selection of key indicators for the trust
The section at the bottom of the organisational summary panel provides quick links to the organisation dashboard
and actions
Regional User View
3 – Tool1 – Background 2 – System 4 – FAQ’s
22 22
Click in any of + or – signs to expand /
contract the section
Summary scores are colour coded and
displayed against each assurance category
• It will initially present the scores against each assurance group, using the navigation users can drill down into detail on specific metrics
Organisational dashboard
3 – Tool1 – Background 2 – System 4 – FAQ’s
23 23
Contribution of this metric to the Assurance Group score
• Opening up different assurance categories allows you to drill down to metric information.
• Clicking on any of the icons provides further information on the metric.
• The icons next to each metrics indicate:• Uploaded: the values displayed have been
directly uploaded from another source• Calculated: the values displayed have been
calculated from other information uploaded.
Actual value of this metric
WeightingThings to note on adding of scores:•If there is only 1 metric then this will be the same as the trigger score (e.g. CIP example)•If there is more than one metric in the trigger then these wont necessarily equal the trigger score due to the weighting
Metrics
3 – Tool1 – Background 2 – System 4 – FAQ’s
24 24
Indicators, Metrics and Thresholds
Workforce
Staff :bed ratios
Staff Type ratios
Staff : Activity Ratios
HV against target
Workforce productivity Skill Mix
Turnover and
RedundancyRetention Temporary
StaffNew
StartersStaff
Diversity
Management numbers
Admin and management:
non admin staff
Clinical: non-clinical staff
Number of consultants/
registrars (FTE) per 10,000 bed
days
Number of qualified nursing,
midwifery and HV staff (FTE)
per 10,000 bed days
Future Staff
Supply
Absence Ratio
Absence by Type
Midwife: birth ratio
MH Consultants: population
Number midwives
acute: community
GP: population
Number of health visitors
HV FTEs: Children under
5
NHS Productivity (output and
quality adjusted)
Number of staff in agenda for
change in bands 1-4 as %
total
Total nursing staff bands 1-4:
5-6
Total nursing staff in bands 5-
6: 7+
Ratio non medical band
7+: non medical 1-6
Turnover excluding
transfers and temporary staff as
% total FTEs
Number of vacancies> 3 months by department
Redundancy Numbers as %
total FTE
Number staff expected to retire
in next 5 years
GP vacancies
Total FTE as % previous year
Total FTE as % level 3 years
ago
New Starters within the last
12 months
Staff gender ratio
Ethnicity
Clinical staff gender ratio
Non-clinical staff gender
ratio
% of staff that are temporary-
Doctors
% of staff that are temporary-Nursing Staff
% of staff that are temporary-Support Staff
Absence ratio as % of total
FTE
% Absence by type
% Sickness by type
Number junior doctors as % total Clinical
staff
Number student nurses
as % total clinical staff
Amount spend on education, training and
research as % total revenue
Denotes a GP –related metric
: denoted a ratio
Target ValueIntervals from Target Value
Intervals from Target ValueIntervals from Target Value
Scale
Range
Med riskHigh risk
Lower riskMed risk
Med riskHigh risk High risk
Lower risk
Metric e.g. Midwife: birth ratio
An indicator could be the number of midwives, or the number of births. The data from these indicators is used to create different metrics.
The metric will be assigned a risk level depending on the threshold. Some thresholds are scales where one extreme is good and the other is bad. Other thresholds are ranges where there is an ideal level and anything either side increases risk. In our example a threshold would be a scale, and less than e.g. 3 midwives per birth is ‘high risk’.
Indicator (590 in total)• The base level of data used to make metrics• This data comes from various source data sets
Metric• Metrics display information about an aspect related to
workforce• Metrics can also be direct (taken directly from one
indicator) or compound (calculated by combining more than one indicator)
Threshold• The threshold is the level at which metrics turn green,
amber or red, indicating the risk associated with each metric
• The threshold is different for each metric and based on the value needed for them to be considered high risk, and turn red, or low risk, and turn green
• The threshold can take the form of a scale or range
Example: Indicators, Metrics and Thresholds
3 – Tool1 – Background 2 – System 4 – FAQ’s
25 25
Score• Each metric within the Assurance Map is given a score
indicating the level of risk (low, medium, high)• These scores add up, once normalised and weighted, to
generate the overall assurance group score and indication of risk
Weighting• Each metric is assigned a weight based on six categories:
• Patient safety• Patient satisfaction• Staff safety• Quality Improvement• Financial control• Staff Health and Wellbeing
• Each category is sub weighted, and the score in each category is used to come to a final score for the metric
Aggregation• Scores can then be aggregated to give a score for each
Trigger, Trigger Group and Assurance Group. Each Trigger and Trigger Group are also weighted
ScoringTrust A
Nurse to bed ratio = 0.1
Nurse to bed ratio
normalised score = -0.8
Nurse to bed ratio rating = Red
(Based on set threshold values)
Calculation
Example:Score, Weighting and Aggregation
3 – Tool1 – Background 2 – System 4 – FAQ’s
26 26
The scoring and weighting of metrics should be seen as an overall process with distinct, time-bound, activities and linkages.
Weighting AggregationAdjustmentScoring
Using established and new thresholds we normalise
threshold values and assign a comparable score to all metrics
within the Assurance Map
Each individual metric score is then weighted by a
defined criteria to enable an hierarchy of importance
to be constructed
All weighted scores can then be subsequently adjusted to account for data coverage and confidence based on statistical analysis. The aim is to account for data quality and quantity problems and show how “reliable” the underlying data
used in metrics is
Scores can then be aggregated to give a score for each Trigger, Trigger
Group and Assurance Group
Things to note: Scores come in three forms• Raw scores provide a base level indication of the perceived risk•Weighted scores also indicate perceived risk, but some metrics are identified a potentially larger impact on the organisational workforce•It is the weighted score, adjusted for data quality so we know which data we can rely on
How the scores add up
3 – Tool1 – Background 2 – System 4 – FAQ’s
27 27
Certain data sources are incomplete for all periods. This could be due to timing or data quality issues. To solve this problem the system will determine values it needs to fill and select from a variety of methods for inferring missing. This can be forward looking or backward looking, using a variety of methods. These include:
•Use last known values•Leave blank•Forecasting - Trends are extrapolated from the data. The type of data set will determine the model/method used to extrapolate. Forecasts for 3 and 12 months are then computed•Pro-rating – Use linear interpolation to estimate the data between two points. This may include using a planned future value compared to an actual, allocating the difference equally across the time periods•Relationships - The data will be examined to reveal relationships between metrics. Relationships derived from this process will then be fed in to forecasting models as inputs•Custom calculations (Not used currently) – Use a custom specified mode, calculation or alternative data source to replace the data
Data source provided with
gaps
Select data where the value is “null” based on comparisons
with the expected data set (e.g. Open periods, actuality)
Data that requires inferring for the
future.
Based on the Indicator Type, chose which approach to use
Use previous value
Use forecast
Etc...
Package is called and data is inferred / deliveredUse replacement
System infers and returns data
Data that requires inferring for the
past.CHOOSE METHOD:
Inference Mechanisms
3 – Tool1 – Background 2 – System 4 – FAQ’s
28 28
Purpose:• Whereas the My Analysis section primarily allows you to
view information within the system, the My Actions component allows you to undertake six different types of actions within the system
Key Actions:• Logs: regional assurers can automate their process of
monthly log recording • Threads: records communications and exchange between
Trusts and regional assurers can be recorded and tracked• Quantitative Indicators: can be recorded, viewed and
edited• Notes: user specific notes can be recorded which can only
be viewed by the author• Flags: alerts and flags can be viewed and overridden.• Plan data: trust plans can be uploaded into the system• Reports: reports can be viewed and searched for my
organisation, date and scoring method
Things to note:•Not all action types are available to all users. Which actions you have access to will depend on the user profile you have.•Within the majority of the My Analysis components there are links to the actions that you may wish to take at any given time.
My Actions
3 – Tool1 – Background 2 – System 4 – FAQ’s
29 29
Purpose of My Resources:• The component provides access to a range of help
resources for the system including:• FAQs• Help files• Training Guides
• The section also provide access to the community forums / discussion groups
Purpose of Administration:• This component provides the SHA with the capability to
control security access to information in the portal based on organisational preferences and reporting agreements
• It also enables the SHA to manage the standardised list of workforce dimensions so that these can be updated should the need arise
• As a user, you will not be able to view the Administer component of the portal
My Resources and Administration
3 – Tool1 – Background 2 – System 4 – FAQ’s
30
Plan DataConsiderations:• Uses year one data within the tool to provide the most robust and detailed information• The 4 planning years are projections and therefore do not have the same degree of
reliability• Given the lead in time for education and training data that is four year ahead is needed to
support education commissioning• Understanding any quality and safety issues associated with the plans is important to
ensure the most robust intelligence• Workforce assurance tool needs to use the most accurate plans available and recognise that
provider owned plans are often more complete for activity data as it reflects multi-commissioners activity levels
3 – Tool1 – Background 2 – System 4 – FAQ’s
31
Freedom of Information (FOI) Requests
Considerations:• The majority of the information used in the tool is already publicly available and
therefore will remain eligible for FOI requests• The planning data that will be within the tool is already held by SHAs and the DH. Access
to the planning data will still only be available to the individual organisation, PCT Commissioning Cluster and SHA.
• The use of the data within the tool will be under the same FOI conditions as if requested from the SHA/ DH at present.
• The tool will not be used to publish data but will used to support workforce assurance which will include improving the quality and safety of workforce plans.
• Most SHAs already use Integrated Performance Dashboards which review plans and performance against plan, the only difference with this tool is the length of the plans reviewed.
3 – Tool1 – Background 2 – System 4 – FAQ’s
32
Foundation Trusts data
Considerations:• The range of data included in the tool for Foundation Trusts will be different to that of Non-
Foundation Trusts. • This issue is much wider than the workforce assurance tool and therefore it is proposed that the
information available is used. • The tool is not intended to collect additional information or intelligence and the project team
will be reviewing the national discussions around quality, safety and Foundation Trusts to ensure it keeps up to date with any developments and continues to use the best available information.
3 – Tool1 – Background 2 – System 4 – FAQ’s
33 33
Accessing the portal• Users access the portal by navigating to the Portal homepage in Internet Explorer• Users will be prompted to enter a username. • Navigate to the relevant URL:
Live Environment: https://wfa.westmidlands.nhs.uk Log in using the username and password provided Note that users are signed out of the portal after 30 minutes of inactivity
Things to note:•An NHS N3 Connection is required to Access the portal•The portal operates over HTTPS to increase security•If problems occur the first ports of call are the Help sections in the My Resources tab of the tool, Training and pre-read material.•For further issues, speak to your local IT team or the Helpdesk – information on this is available in the about section of the portal (My Resources)
Helpdesk: Telephone: 0207 303 8777Email: [email protected]
3 – Tool1 – Background 2 – System 4 – FAQ’s
34
Module 3: part 2User Exercises
To be completed
35
Module 4FAQ’s
36
Questions: are Foundation Trusts included?Will this be of benefit to Foundation Trusts?
•Whilst having a differing status within the NHS, Foundation Trusts face comparable workforce pressures and a need to assure the appropriateness of their workforce.•It is envisaged that FTs will recognise the value that this tool can bring for a nominal investment and use it as a check and balance within their planning processes. Will this be of benefit to Foundation Trusts?
•At a system level, the assurance responsibility remains regardless of FT status, and proposals suggest that if anything this responsibility will be strengthened rather than reduced as a result of any changes under 'Liberating the NHS'•During their remaining period, there is an expectation that SHAs and PCT Commissioners / Clusters will 'hold the ring' on system assurance for all NHS services. Do FTs have different data sets?
•National data sources and public domain information will be utilised to minimise the information requirement on all NHS bodies.•Recognising that differing sets of data will be available for FTs, the system will use the information available to indicate risk. Obviously as the quantity and quality of information decreases so will the ability to effectively highlight risk areas and this will be clearly identified in the tool.
3 – Tool1 – Background 2 – System 4 – FAQ’s
37
Questions: is Primary Care included?Primary care obviously has a large part to play in the assurance of a health system but it also marks the start of a large proportion of patient journeys. This importance is reflected within the assurance model by how we seek to measure referrals and supply of primary care workforce to try and judge the impact on the secondary sector.
Why are you not using QOF as a metric?
• The lack of real variability in QOF scores makes using this data of limited benefit.
How are you linking primary care to the secondary care sector?
• We are looking to link GP practices to secondary providers by mapping practices to commissioners and then identifying which providers these commissioners are “lead” commissioners for.
Why are you concentrating only on GP practice workforce and referrals?
• We think these metrics should give a proxy for supply and coverage in the primary care workforce that can be matched with referral patterns, and subsequent secondary activity, to approximate demand emanating from primary care. We will look for correlations between these data points that can be used in system assurance.
3 – Tool1 – Background 2 – System 4 – FAQ’s
38
Questions: what is the assurance mapThe assurance map is a mechanism for considering potential workforce risks in a structured, logical and data driven manner. The map is hierarchical consisting of a number of categories and levels.
Trig
gers
ba
sed o
n c
urr
ent data
ava
ilable
Overall Assessment
Assurance against plan
Safety InfectionsPreventativeFinance
Organisational Workforce Profile
Staff Experience
Workforce
Staff Survey
Staff turnover and
redundancy numbers
Safeguarding issues
Staff Diversity
Prevalence of Decubitus
ulcers
Patient Survey
PROMs
Retention
Pay of staff
Staff: Activity Ratios
Skill Mix
Health Visitors against targets
Assurance of plan
QIPPPatient
Experience
Absence ratio
Number of Single Sex
Accommodation breaches
Deaths within 30 days of procedure
Data Quality
Financial Performance
Cdifficile
Median time to report incidents
Maternal Mortality
MRSA
Ambulance Response
Times
Negligence Costs
Forecast QIPP Delivery
Immunisations
NPSA Patient Safety
Incidents- Severe harm or
death
HSMR
Staff to bed ratios
Staff type ratios
Workforce Productivity
OthersMortality
Temporary Staff Costs
Smoking Quitters
Changes in strategy/policy
Governance WhistleblowingLeadership movement
Staff governance
Governance management
Financial
Equality & diversity
Clinical governance
Engagement of staff
NHS White Paper
PCT management
cost reductions
NHS tariff changes
Major changes to care
pathways at national\
regional level
Changes in NHS
redundancy arrangements
Freezing of incremental pay
Cases of neglect by staff
Senior doctors moonlighting
Bullying and harassment
Patient safety
Malpractice
Wrongful overtime claims
High numbers of key
leadership posts
Skills gaps
Quality leadership
Talent development
Loss of key clinical leaders
Rapid Turnover Chief Exec/Executive directors
Health work and wellbeing
(Boorman report)
Qualitative Indicators
Total QIPP programme as a % turnover
Pay costs per bed day(Acute)
Income Breakdown
Hours worked against plan
Future Productivity- Level change
Patient Complaints
Quality Outcomes Framework
End of Life Care- deaths at
home
GP Survey Results
Alcohol Related Mortality
Life Expectancy at Birth
VTE Risk Assessment
Hand WashingHealth Check
Clinical Audits
Are clinicians involved in planning?
Focus on Innovation?
Board Capability
Is plan based on evidence?
AQMAR
Finance
Temporary Staff Costs
CIPs
Pay of staff
Absence by group
Workforce
Staff type ratios
Staff: Activity Ratios
Health Visitors against targets
Staff to bed ratios
Workforce Productivity
Skill Mix
Staff turnover and
redundancy numbers
Retention
Staff Diversity
Activity
Referral to Treatment time
Delayed Transfers of
Care
Operations Cancelled
Bed Occupancy
Elective Admissions
A&E Attendances
Daycase and Outpatient
Attendances
Diagnosis 6 week breaches
Readmissions
Education
CPD/ PDP
Training
GMC/NMC re-registration
rates
Underlying Financial Position
Negligence Costs
Health and wellbeing of
staff
Absence by Type
Temporary Staff
Temporary Staff
Maternity Performance
Mental Health Performance
Number of low Value
Procedures
Length of Stay
Activity
Delayed Transfers of
Care
Operations Cancelled
Bed Occupancy
Elective Admissions
A&E Attendances
Daycase and Outpatient
Attendances
Diagnosis 6 week breaches
Readmissions
Maternity Performance
Mental Health Performance
Number of Low Value
Procedures
Length of Stay
Underlying Financial Position
Pay costs per bed day (Acute)
CIPs
Absence ratio
Absence by Type
Future Staff Supply
Future Staff Supply
Referral to Treatment time
Ass
ura
nce
gro
up
Trig
ger
gro
up
Further details about the triggers can be found in the level below by trigger group
(individual metrics)
Pre registration placements as % yearly intake
NPSA Patient Safety
Incidents- Moderate, Low
or No harm
For discussion at a later stage- these form recommended structure for extra,
non data based information
Trigger groups
Triggers
Assurance Categories
Metrics
Workforce
Staff :bed ratios
Staff Type ratios
Staff : Activity Ratios
HV against target
Workforce productivity
Skill MixTurnover
and Redundancy
Retention Temporary Staff
New Starters
Staff Diversity
Management numbers
Admin and management:
non admin staff
Clinical: non-clinical staff
Number of consultants/
registrars (FTE) per 10,000 bed
days
Number of qualified nursing,
midwifery and HV staff (FTE)
per 10,000 bed days
Future Staff
Supply
Absence Ratio
Absence by Type
Midwife: birth ratio
MH Consultants: population
Number midwives
acute: community
GP: population
Number of health visitors
HV FTEs: Children under
5
NHS Productivity (output and
quality adjusted)
Number of staff in agenda for
change in bands 1-4 as %
total
Total nursing staff bands 1-4:
5-6
Total nursing staff in bands 5-
6: 7+
Ratio non medical band
7+: non medical 1-6
Turnover excluding
transfers and temporary staff as
% total FTEs
Number of vacancies> 3 months by department
Redundancy Numbers as %
total FTE
Number staff expected to retire
in next 5 years
GP vacancies
Total FTE as % previous year
Total FTE as % level 3 years
ago
New Starters within the last
12 months
Staff gender ratio
Ethnicity
Clinical staff gender ratio
Non-clinical staff gender
ratio
% of staff that are temporary-
Doctors
% of staff that are temporary-Nursing Staff
% of staff that are temporary-Support Staff
Absence ratio as % of total
FTE
% Absence by type
% Sickness by type
Number junior doctors as % total Clinical
staff
Number student nurses
as % total clinical staff
Amount spend on education, training and
research as % total revenue
Denotes a GP –related metric
: denoted a ratio
Assurance categories: There are four assurance categories that group together trigger groups that show:
1.Assurance of plan (quality of the plan)2.Assurance against plan (variance analysis)3.Organisational workforce profile (general organisational indicators)4.Qualitative Indicators (not metric based but a prompt for areas to look for in an assurance process)
Trigger groups: a collection of similar assurance triggers e.g. workforce or finance
Triggers: a collection of similar metrics / indicators that can be seen as a natural group e.g. Staff Turnover or Cost Improvement Metrics: individual, national, local and bespoke metrics on workforce, quality, safety, finance, activity, etc.
High level
Low level
3 – Tool1 – Background 2 – System 4 – FAQ’s
39
Questions: how were the triggers decidedThe short-list of triggers were decided through consultation with regional reference groups / stakeholders within SHAs. We received a number of datasets from each SHA which had varying levels of detail, quality and confidence in the validity. It is recognised that metrics will be refined and further developed over time as more data becomes available, as reporting requirements change or further research shows that there are other factors or metrics that should be added or removed.
NB. The triggers will be refined and added to as time goes on, and will be subject to ongoing clinical input and validation.
Workforce
Staff :bed ratios
Staff Type ratios
Staff : Activity Ratios
HV against target
Workforce productivity Skill Mix
Turnover and
RedundancyRetention Temporary
StaffNew
StartersStaff
Diversity
Management numbers
Admin and management:
non admin staff
Clinical: non-clinical staff
Number of consultants/
registrars (FTE) per 10,000 bed
days
Number of qualified nursing,
midwifery and HV staff (FTE)
per 10,000 bed days
Future Staff
Supply
Absence Ratio
Absence by Type
Midwife: birth ratio
MH Consultants: population
Number midwives
acute: community
GP: population
Number of health visitors
HV FTEs: Children under
5
NHS Productivity (output and
quality adjusted)
Number of staff in agenda for
change in bands 1-4 as %
total
Total nursing staff bands 1-4:
5-6
Total nursing staff in bands 5-
6: 7+
Ratio non medical band
7+: non medical 1-6
Turnover excluding
transfers and temporary staff as
% total FTEs
Number of vacancies> 3 months by department
Redundancy Numbers as %
total FTE
Number staff expected to retire
in next 5 years
GP vacancies
Total FTE as % previous year
Total FTE as % level 3 years
ago
New Starters within the last
12 months
Staff gender ratio
Ethnicity
Clinical staff gender ratio
Non-clinical staff gender
ratio
% of staff that are temporary-
Doctors
% of staff that are temporary-Nursing Staff
% of staff that are temporary-Support Staff
Absence ratio as % of total
FTE
% Absence by type
% Sickness by type
Number junior doctors as % total Clinical
staff
Number student nurses
as % total clinical staff
Amount spend on education, training and
research as % total revenue
Denotes a GP –related metric
: denoted a ratio
Triggers
Trigger group
Metrics
Refined metrics list on:•Availability of data;•Links to workforce, quality, safety or planning; and•Links to organisational health outcomes
Ran a metrics workshop with key SHA representatives to
agree core set of triggers/metrics
Asked SHAs to review first cut of metrics and comment (using
regional reference groups)
Looked at national recommendations and best
practice
Looked at data available
3 – Tool1 – Background 2 – System 4 – FAQ’s
40
Questions: how granular is the information?The initial metric set for the build will focus on overall organisation-level performance with some speciality/ward/condition level measurement. Where possible, within the constraints of the available data the statistical analysis will look for correlations at a more granular level. Where relationships are identified more granular metrics will be added to the tool.
Activity
•The majority of metrics will measure organisation level performance i.e. referral to treatment times, bed occupancy)
•A smaller sub-set of metrics will look at speciality/condition level activity and performance i.e. LOS for select procedures, maternity performance
Finance
•FIMS data will provide organisation level financial performance details i.e. total pay bill, clinical to non-clinical staff pay, % pay bill spent on overtime
•Non FTs only
Workforce
•Workforce will look at staffing and overall workforce performance at the staff group level (Doctors, Nurses, Midwifes, Health Visitors)
Quality/Safety
•Quality and safety measures will exist largely at the organisation-level given the availability of data i.e. Number of SUIs, NPSA patient incidents
•As more granular data becomes available it may be added to the system
3 – Tool1 – Background 2 – System 4 – FAQ’s
41
Questions: how are data quality issues managed?
Using statistical analysis to provide a data confidence and coverage score will ensure that the accuracy of information is incorporated into the assurance framework and system outputs. Creating a tool that users adopt will drive up data quality.
Data Quality
Data Use
Confidence
An automatically calculated measure of “data quality” and “predictability” of a data set
Coverage
An automatically calculated measure of “completeness” of the data underlying a
metric score.
This can be over time or within peer group
Definitions
Historic experience suggests that as assurers, commissioners and trusts realise the benefits of the system, the data will cleanse and the quality of national and plan data that is monitored will
increase.
3 – Tool1 – Background 2 – System 4 – FAQ’s
42 42
Data is given a score representing confidence in the metrics which have been used to calculate it. Confidence is driven by the coverage and regularity of the data set. Each score is then weighted based upon its relative importance, for example recentness of gaps might be more significant than history length. The worst case scenario score for each indicator, used in a metric, is then taken. Scores are also time-dependent due to the history score and will be based on a rolling 3 year period. Factors that determine data quality are:
Description Rationale Measurement method View
HistoryHow long the data has been collected for.
It has been assumed that data which has been collected for 3 years will be more reliable
Calculate the time between today and the time of the earliest observation.Use this to score the data.
By source
GranularityIs the source data supplied on a monthly, quarterly or yearly basis
More granular data enables more detailed information
Scores allocated for monthly, quarterly and yearly data By source
Recentness of gaps
Whether the gaps in the data are recent or further in the past
More recent gaps will have a more significant impact upon the confidence in the data collection.
The past 3 years of months are allocated scores from 1-36This is multiplied by 1 for any months that have missing data.
By indicator
Number of gaps
How many missing observations are there out of the expected data set.
The more missing data, the poorer the data quality
Comparing actual observations with expected, assigning a score of 1 for each missing observation
By indicator
Gap lengthContinuous sections of missing observations, e.g. Consecutive months.
A large gap size indicates serious concerns with the data quality.
The first gap encountered is scored at 1. A continuous gap from this period is then scored 2 and so on, resetting to 1 if there is an observation in between
By Indicator
FrequencyHow often the gaps occur A high frequency of gaps means that there
are systematic problems with the data as opposed to surprise events
Score 1 for the first gap encountered and do not score continuous gaps from here.
By Indicator
Pattern / regularity
Whether gaps in the data occur at the same time of the year
Gaps map occur for valid business reasons which makes them less concerning than ad hoc gaps in the data
Assign 1 if there is an observation in a month.Subtract this from the previous year’s score. Do this for each year. Add the positive absolute scores together .
By Indicator
Questions: what is the process of adjustment for data quality?3 – Tool1 – Background 2 – System 4 – FAQ’s
43
Questions: who can see what information?
When accessing the system users will be working with data for which they already have access (either because it is within the public domain, is their own data, or because it is information used in current assurance and/or performance management arrangements). The benefit of the system will not be enhanced access to data, rather enhanced access to data that has been coordinated, triangulated and presented in a way that optimises the workforce assurance process.
Providers
Commissioners
System Assurers
1. Plans / scenarios can be “reviewed” in private by the Trust
2. Benchmarking by Trusts will be anonymous – you will see your position and the distribution, but not who the other data points relate to.
3. Ideally, the clinical value of the tool will encourage organisations to increase the role of Directors of Nursing, Medical Directors and Clinicians in decisions relating to workforce planning.
Commissioners will be able to view data related to the providers from whom they commission.
Assurers will have a complete view of the data for those organisations they are responsible for assuring and not those of other regions/areas (nb. this may be flexed dependent on changes to the NCB regions in future)
3 – Tool1 – Background 2 – System 4 – FAQ’s
44
Questions: why is data analytics important?
What additional capabilities will be enabled by data analytics?
•The ability to use forecasting to help identify potential issues before they occur, and enabling a behavioural change in assurance – which should become mostly proactive rather than mostly reactive;
•The ability to challenge accepted wisdom by supplementing expert opinion with robust statistical analysis;
•The ability to accurately analyse trends and cumulative workforce changes to provide a more rounded picture of the potential impact these changes may have on quality and patient safety;
•The ability to produce more sophisticated workforce metrics, and partly avoid the “yes, but it’s more complicated than that” factor;
•The ability to present logical data-driven assumptions around workforce performance benchmarks and expectations; and
•The ability to provide some level of “self-audit” to planners and provide feedback and challenge on their assertions around workforce plans and forecasts.
3 – Tool1 – Background 2 – System 4 – FAQ’s
45
Questions: what happens to the tool after the SHAs?Workforce assurance will take place in the proposed new structure for the NHS. With the challenges facing the NHS, There is a strong rationale for this being done by an overall assurer. This is, of course, to be decided and could be the remit of the National Commissioning Board, Health Education England or Local NHS Education and Training Boards
The tool will provide a robust way of assuring the workforce for commissioners and providers alike.
The tool will provide Commissioning groups/Commissioning Clusters or GP Consortia confidence in current workforce performance and workforce plans, in relation to quality, safety and patient experience.
It would also simplify negotiations around workforce plans and assurance which would condense timescales and potential tensions.
3 – Tool1 – Background 2 – System 4 – FAQ’s