deep-dive review - southwark ccg...deep-dive review: evaluating the ccg’s use of patient...

ENC B

1 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland

The best possible health outcomes for Southwark people

Deep-Dive Review:

Evaluating the CCG’s use of

Patient Experience

Information

Omar Al-Ramadhani Planning and Assurance Manager

ENC B



1. Aim ................................................................................................................................... 3

2. Introduction ..................................................................................................................... 3

3. Sources of Patient Experience Information and Themes ............................................ 5

3.1 Section Summary: Current Information .................................................................................... 5

3.2 Overview of Data ..................................................................................................................... 5

3.3 Nationally-Generated Patient Experience Data ....................................................................... 7

3.4 Locally-Generated Patient Experience Data .......................................................................... 10

4. How the CCG Currently uses Patient Experience Information and how we can

Improve our Approach ......................................................................................................15

4.1 Section Summary – The Use of Patient Experience Information ............................................ 16

4.2 Strategic Planning ................................................................................................................. 17

4.3 Procurement .......................................................................................................................... 19

4.4 Monitoring and Evaluation ..................................................................................................... 20

5. Recommendations.........................................................................................................24

Appendix A - Friends and Family Test.............................................................................25

Appendix B - Sources of Patient Experience Information .............................................28

Appendix C – HRWD Tracker ............................................................................................31

ENC B



1. Aim

This report aims to review how NHS Southwark CCG uses patient experience information to inform

service design and improve the quality and safety of NHS services used by Southwark patients. The

review looks at patient experience information obtained from national and local sources and

additionally considers the gaps in the CCG’s approach. In summary, the review focuses on

answering the following questions:

Is the information available to the CCG fit for purpose and are there any gaps?

How does the CCG collate and analyse patient experience data and to what extent are we

effective in using this information to improve services?

What are the most frequently identified patient experience issues that require improvement?

2. Introduction

Commissioning services that offer a good standard of patient experience has consistently been a

key priority for the CCG. Improving patient experience has been a key objective in the CCG’s

operating, strategic plans and recently published CCG Quality Framework. The CCG also

recognises that good patient experience is a fundamental component of good quality care.

Understanding patients’ views of commissioned services is an essential feature of monitoring the

services we commission.

The CCG’s commitment to improving patient experience is also reflected in national policy. In the

white paper, Equity and Excellence: Liberating the NHS, published in July 20101, the government

made clear its commitment to improving patient experience and creating an NHS that is ‘genuinely

centred on patients and carers’. Patient experience has gained increased recognition as a core

component of good quality care2 with the NHS Outcomes Framework making clear that the

provision of a ‘good experience’ of care for patients is a central goal for both NHS commissioners

and providers.

Patient experience has also been linked to improved patient outcomes. A systematic review of 55

studies in primary care and hospitals found consistent positive associations between patient

experience, patient safety and clinical effectiveness for a wide range of disease areas. This

supports the case for including patient experience as one of the central measurements of quality in

healthcare and underlines its importance3.

In response to heightened national focus on patient experience following the reports into Mid-

Staffordshire; Winterbourne View and Keogh’s review into hospital mortality, the NHS has increased

the production of patient experience information in order to ensure both providers and

commissioners monitor and improve performance. NHS organisations now generate and have

access to a significant volume of data. However, evidence tells us that data is not yet systematically

1 Department of Health. 2010. Equity and Excellence: Liberating the NHS.

2 Department of Health. 2008. High Quality Care for All - NHS Next Stage Review Final Report.

3 The Health Foundation. 2013. Measuring Patient Experience.

ENC B



used by commissioners of NHS services. A recent paper published in the British Medical Journal4

highlights that, whilst multiple sources of patient experience information is available to healthcare

organisations, not enough is being done to use that information to monitor and improve services.

The following sections of the review – sections 3 and 4 – look in further detail at some of the

information that is regularly available to the CCG. These sections look at some of the key themes in

these data and also review how the CCG currently analyses and utilises this information over the

course of the commissioning cycle.

Section 5 concludes the review with a number of practical recommendations for the CCG to take

forward.

Note: This review reflects on and incorporates recommendations made in the CCG Quality

Framework.

4 A. Coulter, L. Locock, S. Ziebland and J. Calabrese. 2014. Collecting data on patient experience is not enough: they

must be used to improve care. The British Medical Journal.

ENC B



3. Sources of Patient Experience Information and Themes

This section looks at the patient experience information currently available to the CCG and

highlights areas where the CCG could make further use of these data sources. Useful data on

patient experience should have the following characteristics:

a) Provides adequate coverage over its commissioned activity

b) Is broad enough to highlight areas that may require improvement and

c) Granular enough to make it possible to drill down to understand issues in more depth.

3.1 Section Summary: Current Information

a) The CCG regularly focuses on national data sources to provide an overview of patients’ experience of acute, mental health, primary care and social care services.

b) The CCG needs to develop an effective way of collecting informal/soft data to identify trends and recurring issues (refer to recommendations 1 and 2 in Section 5).

c) The CCG should make more use of patient focus groups in certain service areas. Focus groups can uncover unanticipated issues that surveys fail to identify and they can be a good way of engaging people who do not respond to surveys; and the most disadvantaged patients (The Health Foundation, 2013). The CCG should access information being shared at hospital patient groups. This can be done by attending meetings or asking for minutes/notes of meetings (refer to recommendations 1 and 5b in Section 5).

d) For both of the above the CCG should consider an approach to conducting qualitative surveys of patients or the completion of focus groups. This approach could consider the role of local voluntary sector groups and/or Healthwatch Southwark as they have links to patients and the community (refer to recommendation1 and 5 in Section 5).

e) There are some gaps where information is not routinely collected/analysed, these include some out of hospital pathways, small providers and out of hours doctors services (refer to recommendation 4 in Section 5).

f) The CCG does not have access to much data that measures patient experience at the interface between services except for data on discharge from acute care. The data we have is mainly restricted to activity at specific providers rather than across pathways. An effective way of gathering this information could be through patient interviews and focus groups. Continuing care could be a good area to start to gather this information as the team have good links with patients and carers (refer to recommendation 5 in Section 5).

3.2 Overview of Data

This Section provides an overview of the types of patient experience information available to

the CCG and highlights a number of themes.

ENC B



England has been a pioneer in measuring patient experience information and was the first country

to introduce a mandatory survey of hospital inpatients in 2002. National survey reporting has

expanded considerably to now include many aspects of care (box 1)5. There is therefore a

significant volume of intelligence and information available to CCGs. Indeed one challenge faced by

commissioners has been to establish systems for the routine collations and interpretation of patient

experience data.

The CCG commissions a wide array of services from multiple providers and so it must be able to

draw upon a varied approach to capturing patient experience information. All data capture methods

have pros and cons, so a combination of styles may be needed to understand issues at a level

which can be used to inform and drive improvements.

5 A. Coulter, L. Locock, S. Ziebland and J. Calabrese. 2014. Collecting data on patient experience is not enough: they

must be used to improve care. The British Medical Journal.

Box 1: Regular national health and social care surveys carried out in England

Care Quality Commission (CQC) national patient experience surveys

General Practice Patient Survey

National Cancer Patient Experience Survey

VOICES survey of bereaved people

Patient reported outcomes of selected surgical procedures (PROMs)

Adult Social Care Survey

Adult Social Care Carers Survey

CQC social care user surveys

ENC B



The diagram below presents a spectrum of patient experience information. The means of collecting

data are grouped into how specific or general the information is and how descriptive it is (see figure

below).

Source: Measuring Patient Experience, The Health Foundation, 2013.

3.3 Nationally-Generated Patient Experience Data

There are a number of nationally directed patient experience surveys that generate data to be used by NHS providers, commissioners and regulators of services. These data and the results at local providers are summarised below and reflect the most recently available data:

Maternity Survey – 2013

King’s College Hospital NHS Foundation Trust (KCH) achieved scores that were comparable with the national average on every question except for ‘being involved enough in decisions about their care during labour and birth’ which the Trust scored below the national average. Guy’s & St. Thomas’ NHS Foundation Trust (GSTT) achieved scores that were comparable with the national average on every question except for two questions relating to staff behaviour. The trust performs below the national average for leaving patients alone at a time that worried them and responding quickly to the call button.

Accident and Emergency Survey – 2012

KCH achieved scores that were comparable with the national average on every question. GSTT achieved scores that were comparable with the national average on every question. The Trust performed better than the national average regarding doctors and nurses listening to what patients had to say, however the Trust performed less favourably against the national average for feeling threatened by other patients or visitors.

ENC B



Cancer Survey – 2012/13

KCH under-performed in most areas of the survey, achieving scores in the lowest 20%. The Trust performed particularly poorly in explaining what was wrong to patients and taking into account patients views while discussing treatment.

GSTT performed better than KCH but the trust does have a number of areas that require improvement.

Inpatient Survey – 2013

KCH achieved scores that were comparable with the national average on most of the questions, however, the Trust underperformed in explaining how patients would be put to sleep and how to control pain and the time spent on waiting lists. The Trust performed particularly poorly for giving patients enough notice about when they were going to be discharged. GSTT did not achieve any scores below the national average and performed well in a number of areas.

Patient Experience Data Sources Used in the Keogh Review

The review carried out by Sir Bruce Keogh into hospital mortality outliers6 focussed on a range of domains that are indicative of the level of quality and safety of care; one of the domains was patient experience. The review focused on a number of publically available sources of information which when analysed together provide a valuable insight into the general performance of providers highlighting areas that could require further investigation. The review examined the following sources of information:

National data sources

CQC Inpatient survey

NHS England Cancer survey

PEAT: Privacy and dignity

PEAT: Environment

PEAT: Food

Ombudsman’s rating

Friends and Family test

Patient voice comments

PEAT = Patient Environment Action Team

Note: KCH and GSTT were not included in the review

Other National Surveys

The Care Quality Commission (CQC) co-ordinate surveys to collect feedback on the experiences of people using a range of health care services provided by the NHS. One of the surveys carried out is

6 NHS England, 2013. Review into the quality of care and treatment provided by 14 hospital trusts in England: overview

report.

ENC B



annual; the adult inpatient survey, however the rest are carried out every 2-3 years. A cancer annual survey is carried out by Quality Health on behalf of NHS England. Other national surveys include:

Outpatient (2003, 2004/5, 2009 and 2011)

A&E (2003, 2004/5, 2008 and 2012)

Maternity (2007, 2010 and 2013)

GP Patient Survey (collected every 6 months, published annually)

Patient Environment Action Team

The Patient Environment Action Team (PEAT) programme has been superseded by the Patient-Led

Assessments of the Care Environment (PLACE) programme. These self-assessments are

undertaken by teams of NHS and private/independent health care providers and include at least 50

per cent members of the public (known as patient assessors). They focus on cleanliness, food,

hydration, and the extent to which the provision of care with privacy and dignity is supported in

clinical and non-clinical services.

Although the assessments are comprehensive, scoring does not differentiate between levels of

performance well enough. The vast majority of hospitals achieve ratings of good/excellent (8 or 9

out 10), which makes it difficult to assess performance and track improvement. As with most

national surveys PLACE surveys are carried out annually.

Friends and Family Test

The Friends and Family Test (FFT) is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. The Friends and Family test is discussed in detail in appendix A.

General Practice Patient Survey

Although the CCG does not commission primary care, it has a statutory duty to assist and support NHS England in securing continuous improvement in the quality of primary care. A source of patient experience information is the national GP Patient Survey which assesses patients’ experiences of the access to and quality of care they receive from their local GPs, dentists and out-of-hours doctor services.

Southwark’s overall survey response rates and satisfaction with GP services and out-of-hours services were similar to the London average.

Patient Voice Comments

Patient Voice Comments measure counts of negative comments adjusted by counts of positive comments about services at a trust. For any source of patient feedback, there will be an expected relationship between the number of positive comments and the number of negative comments across all trusts. For example, the number of negatives would be expected to increase at a given rate as the number of positives increase. Using a statistical procedure called Poisson Regression, results from all trusts are taken to work out what that specific relationship is (i.e. the slope of the regression indicates how quickly negative comments increase relative to positive comments). Then, for each trust, this relationship is applied

ENC B



to the number of positive comments to predict what the number of negatives would be. The departure of the observed number of negatives from the expected (predicted) number determines how much of an outlier a given trust is. If a given trust’s observed count of negative comments is much higher than expected, it can potentially be flagged as a risk by the CQC (if the disparity is extreme and statistically significant). The table below shows that GSTT’s observed negative comments is below expected but King’s negative comments are slightly above expected. The CQC has not identified King’s score as a risk.

Expected Observed

King’s 4.75 5.0

GSTT 4.9 4.0

Source: CQC Intelligent Monitoring Report, 13 March 2014

The information available from the CCG’s main providers is included in appendix B.

Limitations of National Surveys

1. Frequency. The annual surveys can be an insightful source of information on a number of areas of care; however by the time responses are gathered and results published, issues may have already been addressed by the provider or new issues may have occurred which are not captured by the results. Running more regular and focused local surveys would help to monitor services more closely and highlight emerging trends.

2. Granularity. National survey questions are fairly general and do not drill down into specialty or

ward level. This makes the information useful for identifying general issues and trends which could require further investigation. An example of this occurred after a national survey showed that London hospitals were performing poorly in maternity care which Southwark CCG followed up by developing a Commissioning for Quality and Innovation (CQUIN) payment to improve certain areas of maternity care at King’s.

3.4 Locally-Generated Patient Experience Data

Patient Experience Data Generated by Foundation Trust Providers

GSTT produce an annual patient experience report which appraises the trust’s performance in recent national surveys, performance in the trust’s own near time feedback surveys, performance against patient experience CQUINs and a summary of patient advice and liaison service performance. The report also reviews findings from the trust’s patient engagement activities. There is a good mix of quantitative and qualitative information. The report highlights areas of good performance and also areas that require improvement. The issues are discussed generally, however the report could form a good basis for drilling down into areas of concern with the trust and identifying specific wards or specialties that require improvement. King’s produce monthly ‘how are we doing’ (HRWD) surveys (refer to Appendix C for the HRWD tracker) which ask patients questions on patient experience (including those covered by CQUINs) and certain points of delivery. The report used to only present quantitative information; however since August 2013 the report has published all patient comments made at the Trust as well as

ENC B



comments made about the trust at NHS Choices and Patient Opinions. The quantitative data is broken down by specialty which helps provide a good level of detail to the data. The trust also produces a quarterly patient experience report which provides an update on the trust’s performance in national surveys, the HRWD survey, complaints, PALS and a review of the trust’s rating on NHS Choices. GSTT has replaced its annual community services patient experience report with near time surveying. This provides more regular data and is currently reported quarterly for ten services. The trust is trying to increase the accessibility of the survey in a number of ways including; enhancing the domiciliary telephone process; making surveys more user friendly for children and those with learning disabilities and providing support to help capture the views of more vulnerable patients.

GSTT Community services

District Nursing

Foot Health

Health Visiting

Intermediate Care

Occupational Therapy

Paediatrics

Physiotherapy

Reproductive and Sexual Health

Speech and Language Therapy

Specialist Regional Rehabilitation

There is a large variation in the number of questionnaires submitted for each service with the most being around 1,000 for Foot Health and as little as nine for Specialist Regional Rehabilitation. The surveys do not provide any qualitative data, however they could be effective at identifying high level problem areas. So far the information has not resulted in service improvements because the data has not been able to illuminate any areas that require improvement. Options being considered to improve the surveys include; themed questions on specific areas and bespoke questions for each service. Commissioners are currently designing a survey for district nursing with bespoke questions that will attempt to cover the integration of services. SLaM’s patient experience reporting is centred around the trust’s Patient Experience Data Information Centre (PEDIC) reports which present quantitative data for inpatients, outpatients and each clinical academic group (CAG). The questions in the report have been agreed by commissioners with the trust. The last report (Q3 13/14) is detailed enough to monitor performance over time and initiate discussions with the trust on areas that require improvement. A patient experience survey was carried out by the Southwark IAPT service between January 2013 and December 2013 receiving over 1,000 responses. The survey asked five questions and allowed patients to make comments and so provides both quantitative and qualitative information. On the whole, patients rated the service highly.

ENC B



Local Provider Themes

The table below presents areas of patient experience which have been highlighted as under performing from the trust reports mentioned above.

GSTT Acute

Inpatients

Receipt and explanation of the ward pack

Quality of the food

Medication side effects

Noise at night

Cleanliness of toilets and bathrooms

Outpatients

Choice of appointment times

Being kept informed of appointment wait in clinic

Addressing worries and fears

Knowing who to contact

Delays in appointment starting time

King’s

All areas

Being told how long you would wait in outpatients

Complaints response times

Noise at night from staff

SLaM

Inpatients

Receiving a copy of care/recovery plan and jointly producing the plan

Explanation of possible medication side effects

Offered a meeting with the ward pharmacist

Restraint and injected medications used as a last resort with minimum use of force?

GSTT Community Services

NA

Receiving a copy of care/recovery plan and jointly producing the plan

Knowing how to make a complaint

Offered a crisis plan for emergency mental health situations

Being given a full assessment after accessing crisis support at SLaM

ENC B



CCG Engagement

NHS Southwark CCG regularly captures patient experience information from the Engagement and

Patient Experience Committee (EPEC), a sub-Committee of the Governing Body and local Patient

Participation Groups (PPGs). The CCG also captures information from large ad hoc patient

engagement events and through a number of service user groups (Discussed further in Section 4).

EPEC and PPG Feedback

Concerns raised at EPEC and locality PPGs are recorded in meeting minutes and are also

communicated to lead officers and the CCG Governing Body through the CCG’s quarterly quality

report. Issues are also raised directly by the Head of Membership and Engagement with

commissioners, however this is currently done informally.

Issues that have been captured include:

Waiting times for inpatient treatment

Outpatient clinic waiting times

Cancelled outpatient and inpatient appointments

Discharge delays from hospital

Referrals being lost

Quality of follow up and after care particularly after cancer treatment.

Timeliness of discharge information

Communication between secondary and primary care.

The CCG can potentially collect large amounts of informative soft data through engagement events

and patient forums, however soft data is not always formally published and so the CCG’s access

and use of it can be limited. Feedback collected from engagement events or patient forums are not

always written up and analysed, which risks information being lost.

Out of Hospital Pathways

Most out of hospital pathway services are contracted to carry out annual patient experience

surveys. The dermatology service returns an annual survey and King’s and GSTT are contracted to

survey patients in the Musculoskeletal Clinical Assessment and Treatment Service (MCATS).

Some out of hospital pathway services (for example the diabetes service) return monthly patient

surveys, however commissioners have highlighted that they don’t have the capacity to review the

data on a regular basis but will analyse the data during a service review.

Feedback indicates that beneficial data has come from conversations with patients during visits (for

example the diabetes quality visits) and by engaging patient groups and forums. The Southwark

and Lambeth diabetes forum has been a good source of information as it represents a diverse and

fairly representative group of patients. Interviewing patients and running focus groups can be

ENC B



resource intensive so it has been suggested that collaborating with voluntary services who

already have links with patients may be an effective way of gathering a rich perspective on patient

experience.

Patient experience information across pathways

The CCG lacks access to patient experience information that tracks patient experience across

pathways and the transfer between health and care services. This gap in information has been

recognised nationally and prompted five leading healthcare organisations to initiate an 18 month

project aimed at capturing older people’s experiences of care coordination. The project will develop

a survey tool to measure how older people and service users experience care when it is delivered

by multiple organisations. It is hoped that the final tool set will be available for use in the summer of

20157.

Sources of Information the CCG does not routinely utilise

There are a number of sources of information that the CCG does not routinely use, these are listed

below:

Hospital patient groups – There are many patient groups at King’s that meet to share feedback on

services; these include

King’s In Conversation

King’s In Your Shoes

King’s Maternity Services Liaison Committee

King’s Cancer Voices

Online chat forums and social media – These can be a useful source of patient comments about

services in Southwark. The CCG has used the SE1 online forum to ask Southwark people for their

experiences of using the urgent care centre in June 2013. The discussion generated 3 pages of

comments and allowed the CCG to engage with members of the forum to ask follow up questions.

The CCG has also used the Myhealth London chat forum facility as part of the Call to Action

campaign but the topic did not generate any discussion.

The CCG has been using Twitter to publicise its engagement events and activities. Tweets have

mainly focused on drawing peoples’ attention to the CCG’s website and not for direct engagement.

The Twitter account has 3,766 followers.

7 The Nuffield Trust: New project launched to explore how well health and social care services are coordinated.

http://www.nuffieldtrust.org.uk/media-centre/press-releases/new-project-launched-explore-how-well-health-and-social-care-services. Accessed 14 August 2014.

http://www.nuffieldtrust.org.uk/media-centre/press-releases/new-project-launched-explore-how-well-health-and-social-care-services

http://www.nuffieldtrust.org.uk/media-centre/press-releases/new-project-launched-explore-how-well-health-and-social-care-services

ENC B



4. How the CCG Currently uses Patient Experience Information

and how we can Improve our Approach

This section will review how patient experience information has been used by the CCG to

commission services and how the use of information could be improved.

The CCG’s work can be summarised by the commissioning cycle below:

Involving patients in the commissioning cycle and ensuring services are commissioned to improve

patient experience can lead to more efficient and cost effective pathways and ones that lead to

better outcomes.

ENC B



4.1 Section Summary – The Use of Patient Experience Information

a) The CCG has been effective at conducting large scale engagement events like those that inform the CCG’s commissioning intentions and the recent Call to Action event. However, we are aware that some feedback indicates that engagement events are not held early enough in the planning process to ensure that decisions are genuinely informed by patient views. There is a feeling that some engagement events and forums have been used to endorse decisions that have already been made.

b) When carried out in good time, engagement events have been an effective way of gathering

information to inform strategic decisions. It is essential that Commissioners work with the Communications and Engagement team to feedback to patients and participants in our engagement to demonstrate where their involvement has helped to shape our plans.

c) The CCG should undertake a different approach to focussed engagement. Our experience is that engagement in small forums and discussion groups work best when it includes expert patients or current and ex-users. Wherever possible the CCG should focus engagement on issues that patients are able to offer valuable insights and opinions (refer to recommendation 1 in Section 5).

d) The CCG has been effective in driving service improvements through the use of CQUINs.

However without the use of CQUINs (and quality accounts at King’s) it is difficult to engage providers to measure additional areas of patient experience and implement improvement plans. Clinical Quality Review Groups (CQRGs) are used as a way of exerting influence to encourage providers to improve on certain areas of patient experience (refer to recommendation 3 in Section 5).

e) There are good examples of how triangulation of national and local data (King’s maternity care)

and the triangulation of patient experience information and complaints information (mental health) has been effective in driving improvements. This approach should be enhanced as part of the CCG’s assurance of commissioned providers (refer to recommendation 2 in Section 5).

ENC B



4.2 Strategic Planning

Examples of Good Practice

The CCG has been effective at carrying out large engagement events to gather patient experience

information. The recent Call to Action event for example was attended by around 70 people and

produced a wealth of information that was captured in a well written report.

Recently, the CCG used a Survey Monkey to gather information on the experience of users of the

talking therapy services to inform the development of the new service. The survey returned over 60

responses and asked a number of questions across a range of themes. The survey also provided

free text boxes to allow for more detailed feedback. Widely accessible electronic surveys like this

can provide valuable benefits particularly if they are coupled with survey methods aimed at hard to

reach and the most vulnerable patient groups.

Services like Survey Monkey allow for:

1) Wide coverage and easy access.

2) Eliminates manual input of survey responses.

3) Real time data charts and analysis.

The findings from the survey confirmed that patients wanted access to the service after 6 pm, they

wanted the choice of a range of languages and wanted a more even number of male and female

counsellors.

There are a number of additional examples of when patients have made a valuable contribution to more focussed pieces of work, like in the review of muskuloskeletal physiotherapy services. Engagement here involved a workshop of current users to inform potential changes to the service (box 2).

Box 2: Provision of physiotherapy and osteopathy services.

In early 2013 the provision of physiotherapy and osteopathy in Southwark was reviewed.

The review found that as well as the provision of physiotherapy at hospitals, several GP

surgeries also provided physiotherapy or osteopathy services.

A consultation engagement event was held with patients on Wednesday 25 September.

14 patients attended, most of who had used a non-core funded Osteopathy Service at

their local practice and/or Physiotherapy Services at various sites. Discussions at the

event focused on osteopathy in particular as a number of patients were concerned about

changes to their current practice based osteopathy services. Although the CCG had

decided that a change to the service was necessary to provide a more equitable and

evidence based service; patients were engaged on a number of options on how the

future service may look.

ENC B



The Continuing Care team has worked to support patient representatives during the development of adult personal health budgets (Box 3).

Areas for Improvement

The CCG has generally been less effective at smaller scale engagement of patients at the planning stage of the commissioning cycle. CCG leads have highlighted that on occasion small scale engagement events are not organised at the very beginning of the planning cycle and are carried out to justify strategic decisions that have already been made. It can be difficult for lay people to make valuable contributions at this stage because:

They are sometimes not well informed about the case for change.

Strategic topics are not presented in a way that are easily understood.

The scope of some service changes are sometimes limited by centrally mandated specifications and the need to achieve national targets which limits the scope of engagement.

A number of CCG leads have suggested that asking patients to focus on one or two areas that really benefit from a patient perspective, like the production of information leaflets, may be more beneficial than engaging people on high-level technical issues.

Box 3: Adult Personal Health Budgets The continuing care team tried to overcome these issues when inviting patients to sit on a

steering group to oversee implementation of adult personal health budgets (PHBs). They

chose one patient who was representative of wider groups and one patient who had lived

experience of holding a PHB. To ensure the lay members made useful contributions they

were kept well informed of relevant issues outside of steering group meetings and

additional effort was made to ensure communication was always in a clear format.

ENC B



4.3 Procurement


There are examples of where the CCG has used patient experience information and engagement to

inform procurement decisions.

The Talking Therapies procurement will involve two service users on its steering group and a

representative from Healthwatch Southwark. The process will require a significant time commitment

from those involved as there are monthly meetings (potentially moving to fortnightly) and the

interview process has been planned to take place over a week. The service users are being

supported to take part by receiving technical briefings and receiving compensation.

For mental health services, patients’ involvement in procuring services has been limited. There have

been a couple of examples of children being involved at this stage. Children fed back their views as

part of the 1000 voices initiative and were also involved in helping select early health providers. The

Southwark Youth Council (SYC) and the looked after children user group have been involved in

designing and procuring services (Box 4).


For out of hospital pathways, the general approach is to engage patients through workshops to

discuss procurement options. However, for some services a patient representative from EPEC has

sat on procurement panels even though they are not a current or ex-user of the service (Box 5).

Box 4: Southwark Youth Council Southwark Youth Council (SYC) is a voice for young people run by young people. Young people are redesigning public services and influencing how neighbourhoods are improved. The SYC has been proactive in various decision making activities across the borough. The main members of SYC are young people living, attending school, and working in Southwark. They hold events and workshops as a way for young people to express their views and for SYC to take those views to the decision makers.

Box 5: Procurement of Out of Hospital Pathways Commissioners tried to encourage service users of the ENT and dermatology services to

be involved in the procurement of the new services. However, it was difficult to generate

interest from patients because involvement could require a level of commitment that was

beyond most patients particularly as meetings take place during working hours.

In February 2013, the procurement processes of the ENT and dermatology services were

presented at EPEC. It was agreed at the meeting that there should be a patient

representative present on both procurement panels and both of these representatives

were members of EPEC.

ENC B



4.4 Monitoring and Evaluation


There are a number of good examples of where patient experience information is being used to

monitor and evaluate services.

At King’s, commissioners monitor patient experience performance through monthly HRWD reports

and quarterly Quality and Governance reports and discuss issues further at monthly CQRG

meetings. Working jointly with King’s, commissioners have used patient experience data to initiate

service improvements through the development of CQUINs.

Commissioners have been able to identify areas for service improvement at King’s through patient

experience data and the trust has worked jointly with commissioners to develop CQUINs to improve

performance. CQUINs can be an effective way of improving performance temporarily, however if the

improvements are primarily driven by incentives then there is a risk that performance could

Box 6: King’s Maternity service

In 2011/12, a study of maternity services in England showed that London had one of the

highest maternity death rates; was double the national average and that women in

London were found to be the least satisfied with the care that they received during labour

and birth. King’s HRWD surveys also showed that patient experience was below target

for maternity services. This prompted commissioners to work with the trust to develop a

CQUIN for 2012/13 to improve performance, which did result in an improvement in

performance and the achievement of most of the CQUIN targets.

Box 7: King’s Worst Wards

In 2011/12 commissioners and King’s used the data that informs the HRWD surveys to

identify the worst performing wards. A CQUIN was introduced in 2012/13 to incentivise

performance improvement in those wards and resulted in improvement in patient

experience and achievement of all CQUIN targets.

Box 8: King’s Outpatients

In 2012/13, a CQUIN was developed with the trust to measure and improve patient

experience in specific areas of outpatients that had been identified as requiring

improvement from local HRWD survey results. The areas were identified from survey

results in 2011/12 and a number of indicators were developed to track and improve

performance in areas identified as needing the most improvement.

ENC B



deteriorate following removal of the incentive. Inserting penalties into contracts is another potential

approach; however it has proven difficult to insert penalties for indicators that are not nationally

mandated making it difficult to embed quality improvements contractually.

The CSU works with King’s to improve areas of patient experience by discussing patient experience

issues covered in quarterly quality reports at CQRG meetings. It is difficult to quantify the

effectiveness of this approach on improving patient experience.

Quality Accounts

King’s annual Quality Accounts summarise their performance and improvements against quality

priorities and objectives set by themselves for 2013-2014. It also outlines those they have agreed

for the coming year 2014-2015. The main domains of the report cover patient safety, clinical

effectiveness and patient experience. The two main areas of development under patient experience

for 2013/14 were improving outpatient experience and improving patient experience of discharge.

1. Improving outpatient experience in Suite 1

The aims of 13/14 were to agree which 5 questions are most in need of improvement within

Suite 1, develop action plans and agree improvement target percentage for Q2, 3 and 4. The

trust also aimed to achieve a decrease in patient complaints relating to the outpatient

department.

2. Improving patient experience of discharge

The aims of 13/14 were to improve trust-wide patient satisfaction in two key areas; providing

information about medication after discharge and providing patients with information on

what to do and who to contact if they have a concern after discharge. These are questions in

the HRWD local surveys and so improvement could be tracked on a monthly basis.

Mental Health - Examples of effective use of patient experience information

The CCG Mental Health Commissioning Team regularly engage with patient forums and use the

information to feedback to services for improvement. Some of the common issues that have been

raised include day centre provision, inpatient ward food, inpatient ward cleanliness and staff

attitude.

a) Day centre provision has been a particularly common topic of discussion over the last 12

months coupled with the introduction of personal budgets. Council members have voiced

frustration that they have not been kept up-to-date with latest information which led to project

leads organising a news letter for service users.

b) Comments on the quality and quantity of food on wards are fed directly back to the SLaM ward

manager who in turn picks up the issue with the hospitality contract holders.

ENC B



Any improvements are fed back to the group at the following meeting.

The CCG Mental Health Commissioning Team monitor performance at SLaM through its PEDIC

reports. Areas of concern highlighted by PEDIC survey results are raised with the trust through

commissioner responses and are discussed at quality meetings with the trust.

The CCG Mental Health Commissioning Team also tries to triangulate patient experience

information with complaints reports and again feeds back concerns in commissioner responses.

Commissioners have also conducted ward inspections at SLaM and have picked up a number of

issues that can affect patient experience from ward staff. These have been raised at quality

meetings with the trust.


The triangulation of patient experience information with other quality and safety information has

been improving. The quarterly CCG Quality Report has included some information on patient

experience and information from the CCG’s engagement structure and public engagement events;

this is a good start, however future reports should include more patient experience information to

allow for more consistent and wider triangulation.

Box 10: SLaM patient experience CQUIN

In 2013/14, a patient experience CQUIN was included in the SLaM contract. The CQUIN

involved developing patient focus groups to feedback on areas that require improvement.

The issues highlighted were staff attitude, feeling respected by staff and the environment.

SLaM then developed action plans to improve these areas and this was monitored by the

CCG mental health team. In Q4, a patient focus group met to review progress against

improvement plans and acknowledged that there had been improvement in all the areas

highlighted at the start of the year.

In 2014/15, commissioners requested that SLaM become more pro-active in asking for

patient feedback. It was agreed that the trust be more proactive in asking patients if they

would like to feedback on their experiences at discharge on an ipad. The ipad also gave

patients the opportunity to fill in PEDIC surveys and write comments on their experiences.

Box 9: Southwark Mind

NHS Southwark CCG commission Lambeth and Southwark Mind to run the Southwark

User Council. All members are current or previous service users and represent

commissioned services and visit their service on a regular basis asking for feedback from

current service users. Feedback is then discussed at the Council monthly meetings

(closed for the first 2 hours) and then open with Commissioners and Service providers in

attendance for the next 1.5 hours. Two members of the council sit on the Mental Health

Programme Management Group.

ENC B



CQRGs review patient experience information and this provides an ideal opportunity to triangulate

patient experience information with other quality and safety information. CQRG leads report back to

the Integrated Governance and Performance Committee (IGP) and Governing Body on the main

quality and safety concerns at providers, however there isn’t always a specific focus on patient

experience information. If more patient experience information is reported back, commissioners

could then add their insights to the data and enhance the performance monitoring process.

Organising and Managing the Data

Commissioners have fed back that there is a lack of capacity and a system to collect, analyse and

triangulate data effectively.

Quality Analysis Database

Birmingham CCG has been developing a system that they hope will revolutionise the way they use

patient experience information. They are developing a Quality Analysis Database (QUAND) which

will eventually form a substantial central repository for all the CCG’s patient experience data. They

are designing the system to automatically extract information from national databases like NHS

Choices and Patient Opinion and from local databases like the primary care patient experience

module, Datix. They are also trying to triangulate this data with quality and safety indicators like falls

and pressure ulcers and so the system will be able to automatically extract data from the Strategic

Executive Information System (STEIS).

The system will also allow users to input information manually making it capable of capturing soft

data. Data will be broken down to a granular level allowing users to search at provider, specialty

and even ward level to try and identify trends.

Insights Dashboard

London CCGs are considering procuring a London based patient experience dashboard that will

bring together a wide array of publically available information covering hospitals and GP practices.

There are three different subscription levels available for the CCG to choose from - subscription

levels offer varying degrees of detail and bespoke information. Southwark CCG managers will be

attending the London CCG Engagement Leads meeting on 29 September to review subscription

options.

ENC B



5. Recommendations

1) The CCG could further benefit from the use of patient experience information by triangulating national and local information and by triangulating patient experience information with complaints and quality and safety information. This could provide a more comprehensive level of intelligence which could help commissioners identify trends. Commissioners should consider broadening the source of information to include more frequent use of online forums; and reports and intelligence from organisations that work closely with patients like charities and Healthwatch Southwark.

To allow for effective triangulation, the CCG needs to capture more qualitative data to add greater depth to the quantitative data we currently collate. This can be through issues identified in the CCG Quality Report, in the form of written feedback, local ‘soft’ intelligence captured from engagement with patients. The CCG should consider the resource implications of this development.

2) CCG officers should develop quarterly briefings to CQRG leads, with intelligence gained from a triangulated approach to quality and safety. This will allow CQRG leads to monitor performance with providers more effectively and support them to improve services.

3) Commissioners should further encourage providers (particularly large providers) to design and implement improvement plans to enhance patient experience in specific areas of underperformance. Providers are not keen to engage on this in the absence of financial incentives so a collaborative approach overseen by CQRG meetings could be developed.

4) The CCG should work with services that do not currently provide patient experience information to develop appropriate means of capturing information. The regular capture of patient experience and the implementation of improvement plans should be included in all provider contracts.

5) Commissioners should consider ways of understanding in greater detail patients, carers and their families’ experience of care across pathways and not just at provider sites.

a. The CCG should consider commissioning broad-based surveys of patients to understand from a wider sample more about their experience of the care they have recently received. This would need to look at specific issues that we know are important (e.g. the discharge process; compassion of nursing care etc.).

b. A project should be initiated to understand in greater depth some of the issues facing patients more likely to use health services. Patients with Continuing Healthcare packages of care may be a useful cohort to work with in the first instance. The CCG’s continuing care team works closely with individual patients and their carers and could conduct in-depth interviews to understand in more detail patients’ experiences.

6) The CCG should ensure that patient representatives are involved in all clinical procurements, including procurement panels. Patient representatives can help the CCG identify innovation and good practice and identify providers who meet the needs of patients. For this to happen; they should be experienced patients or carers in the specific area and be offered support by the CCG to fulfil this role.

ENC B



Appendix A - Friends and Family Test

The Friends and Family Test (FFT) is a single question survey which asks patients whether they

would recommend the NHS service they have received to friends and family who need similar

treatment or care. The FFT was rolled-out across the NHS following recommendations made by the

Nursing and Care Quality Forum (NCQF) to the Prime Minister.

Initially, the test had two main aims as defined by the Department of Health:

To provide a comparable performance measure across providers

An improvement tool to enhance understanding about where and how improvements could be made.

A recent review of the FFT by NHS England endorsed the tests potential to facilitate service

improvement; however it did recognise its limitation in providing a comparable performance

measure across providers.

FFT was initially rolled out for providers of NHS funded acute services for inpatients (including

independent sector organisations that provide acute NHS services) and patients discharged from

A&E (type 1 & 2) from April 2013. As of 1 October 2013 the survey was extended to include all

women of any age who use NHS funded maternity services and from 1 December 2014 it will be

available in GP practices. The test will be rolled out even further next year; from January 2015 it

will be rolled out in mental health and community services and from 1 April 2015, it will be

expanded to NHS dental practices, ambulance services, patient transport services, acute

hospitals outpatients and day cases.

The 2014/15 National CQUIN

FFT is part of the 2014/15 national CQUIN which incentivises providers to improve performance.

The 2014/15 CQUIN will incentivise the following:

1. Early implementation of the test in outpatient and day case departments by 1 October 2014.

2. Increasing or maintaining response rates in acute inpatient services.

i. A baseline response rate in Q1 of at least 25 per cent and by Q4 a response rate that is

both (a) higher than the response rate for Q1 and (b) 30 per cent or over; or

ii. Maintaining a response rate that is over 30 per cent.

3. Increasing or maintaining response rates in A&E.

i. A baseline response rate of at least 15 per cent and by Q4 a response rate that is both (a)

higher than the response rate for Q1 and (b) 20 per cent or over; or

ii. Maintaining a response rate that is over 20 per cent.

ENC B



The Question

In its current form, the test asks the following standardised question: “How likely are you to

recommend our ward/A&E department to friends and family if they needed similar care or

treatment?” Patients will use a descriptive six-point response scale to answer the questions with the

following response categories:

1 Extremely likely

2 Likely

3 Neither likely nor unlikely

4 Unlikely

5 Extremely unlikely

6 Don’t know

Friends and Family Test results are calculated using underlying “Net Promoter Score” methodology.

All patients within the target groups will be given the opportunity to respond to the Friends and

Family Test. Patients must be surveyed at, or within 48 hours of, discharge.

Scoring Methodology

The following example explains how the FFT score is calculated. Assume that a trust records the

following results.

Total number of responses

Extremely likely 114

Likely 54

Neither/nor 24

Unlikely 2

Extremely unlikely 6

Total 200

ENC B



The FFT score is calculated by the following formula:

Minus

Friends and Family Test Score: 57 – 16 = 41

A common query regarding the scoring methodology is, why those that respond "likely" are

excluded from the calculation?

FFT is meant to drive excellence within patient experience so, if a patient responds with ‘likely’ then

they will not have had the best experience. However, they are not excluded from the calculation,

because they are included in the denominator. If a patient chooses to respond “likely” instead of a

less positive category, that will lead to a higher score. If they choose to respond “likely” instead of

“extremely likely” that will lead to a lower score. Likely responses therefore have an important

influence on the resulting score.

Proportion of respondents who

would be extremely likely to

recommend (response: “extremely

likely”)

114 ÷ 200 (x 100) = 57

Proportion of respondents who would

not recommend (response: “neither

likely nor unlikely”, “unlikely” &

“extremely unlikely”)

24+2+6) = 32 ÷ 200 (x 100) = 16



Appendix B - Sources of Patient Experience Information

Source/Report Detail Source

National Voices National coalition of health and social care charities in England. Focus on patient centred care

http://www.nationalvoices.org.uk/

King’s How Are We Doing (HWRD)

Opportunity for patients to fill in surveys on many services areas: Outpatients, Emergency Department, Emergency Dental, Early Pregnancy Unit, Maternity Services, Dental Outpatient Services, Inpatients and Day surgery patients. Carers, relatives, friends or visitors can fill in How are we doing? Comments and Suggestions cards. King’s produce monthly reports detailing results.

Jessica Bush – King’s Head of Engagement and Patient Experience. [email protected]

King’s In Conversation

Listening events with patients regarding the following questions: Are patients always our first priority? Would you recommend our hospitals as places to receive care? What do you think we could do better?

http://www.kch.nhs.uk/about/get-involved/groups-events/kings-in-conversation

King’s In Your Shoes Opportunity for King’s staff to listen to patient views. http://www.kch.nhs.uk/about/get-involved/groups-events/in-your-shoes

King’s Maternity Services Liaison Committee

The MSLC is a group that includes mums as well as midwives, doctors and other health professionals. The group meets about every two months.

http://www.kch.nhs.uk/about/get-involved/groups-events/maternity-services-liaison-committee

King’s Cancer Voices A group made up of patients, relatives and carers affected by cancer and health professionals.

http://www.kch.nhs.uk/about/get-involved/groups-events/cancer-voices

King’s Have Your Say Repository of patient comments http://www.kch.nhs.uk/about/haveyoursay

http://www.nationalvoices.org.uk/

mailto:[email protected]




http://www.kch.nhs.uk/about/get-involved/groups-events/in-your-shoes

http://www.kch.nhs.uk/about/get-involved/groups-events/in-your-shoes






http://www.kch.nhs.uk/about/haveyoursay




GSTT Feedback Opportunity for patients to fill in surveys on Friends and Family test, general surveys, service-specific surveys and Community services.

http://www.guysandstthomas.nhs.uk/patients-and-visitors/feedback/your-feedback.aspx

GSTT 2012-13 annual Patient Experience and Engagement Report

Provides an overview of the work that has taken place during 2012-13

GSTT Board papers http://www.guysandstthomas.nhs.uk/about-us/publications/board-papers/2013-board-papers.aspx#na

The GP Patient Survey GP survey results available at CCG and individual practice level. http://practicetool.gp-patient.co.uk/Ccg/Search?id2=southwark%20ccg&index=0

CQC National Maternity Survey 2013

Key findings from the 2013 national survey of women’s experiences of maternity care.

http://www.cqc.org.uk/public/publications/surveys/maternity-services-survey-2013

CQC National A&E Survey 2012

Asks people about their experiences of A&E departments in 2012. http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/accident-and-emergency-2012

CQC National Inpatient Survey 2012

Asks patients who were admitted to hospital services about their experiences in 2012.

http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/inpatient-survey-2012

CQC National Outpatient Survey 2011

Asks patients about their views on their most recent visit to an outpatient department.

http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/outpatient-survey-2011



http://www.guysandstthomas.nhs.uk/about-us/publications/board-papers/2013-board-papers.aspx#na



http://practicetool.gp-patient.co.uk/Ccg/Search?id2=southwark%20ccg&index=0





http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/accident-and-emergency-2012












CQC Community Mental Health Survey 2013

Asks patients about their experiences of receiving community mental health services.

http://www.cqc.org.uk/public/publications/surveys/community-mental-health-survey-2013

NHS England National Cancer Patient Experience Programme

Asks patients about their experiences of receiving cancer care. http://www.quality-health.co.uk/surveys/national-cancer-patient-experience-survey

Local quarterly Community surveys

Includes; Children occupational therapy, children physiotherapy, children speech and language therapy, District nursing, foot health, health visiting, paediatrics and reproductive and sexual health

Jean Young, Meridian Desktop

Friends and Family Test

The Friends and Family Test is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. Currently the test covers;

i) A&E ii) Inpatients iii) Maternity

http://www.england.nhs.uk/statistics/statistical-work-areas/friends-and-family-test/friends-and-family-test-data/

SLaM PEDIC surveys and reports

There are a number of patient experience surveys carried out by SLaM which cover; outpatients, inpatients and trustwide.

Southwark CCG Mental Health team

Cancer Patient Experience Survey: Insight Report and League Table 2012–13

The 2012–13 National Cancer Patient Experience Survey (CPES) captures a picture of progress among hospital trusts. London trusts are amongst the worst performing in the country

http://www.macmillan.org.uk/Documents/AboutUs/Research/Keystats/2013CPESInsightBriefingFINAL.pdf




http://www.quality-health.co.uk/surveys/national-cancer-patient-experience-survey











Appendix C – HRWD Tracker