pals core national standards and evaluation...

1

PALS CORE NATIONAL STANDARDS AND

EVALUATION FRAMEWORK

Assessing performance against

National Core Standards for PALS

2

DH INFORMATION READER BOX

Policy EstatesHR / Workforce Performance Management IM & TPlanning FinanceClinical Partnership Working

Document Purpose Best Practice Guidance

ROCR Ref: 0 Gateway Ref: 1757

Title

Author

Publication DateTarget Audience

Circulation List

Description

Cross RefSuperceded Docs

Action Required

TimingContact Details

For Recipients Use

Karen InnsPatient and Public Involvement

Karen [email protected] 254 6109

5CO5

LS2 7UEQH

#VALUE!

These standards are issued by the DH and set out the minimum service which should be delivered by PALS. The standards enable PALS to assess performance and benchmark against the standards in order to develop the service provided in a way that is consistent with other PALS nationally.

N/A

N/A

To use the standards to measure performance and

Supporting the Implementation of PALS: A Resource Pack0

develop service provided

PALS EVALUATION FRAMEWORK:

Department of Health/PALS National Development Group

SHA PPI Leads and PALS Managers30 Jul 2003

Assessing performance against National Core Standards for PALS

3

Contents

Section 1: Introduction 4 Section 2: Planning your evaluation 7 Section 3: Collecting local information 10

Section 4: Assessing performance against the National Core Standards for PALS 12

Section 5: Assessing performance against other measures 32

Section 6: Planning to consult other stakeholders 33 Section 7: Writing your report 37 APPENDICES Appendix 1 - Harrogate Health Care NHS (Acute) Trust 39 Appendix 2 - South West Yorkshire Mental Health NHS Trust 43

Appendix 3 - London PALS pathfinder sites 55 Appendix 4 - University College London Hospitals NHS Trust 65 Appendix 5 - Wakefield West Primary Care NHS Trust 69 Appendix 6 - Hull and East Yorkshire Hospitals NHS Trust 78

4

SECTION 1: INTRODUCTION The core functions of PALS The core functions of PALS are to1:

• Be identifiable and accessible to patients, their carers, friends and families • Provide on the spot help in every Trust with the power to negotiate

immediate solutions or speedy resolution of problems. • Act as a gateway to appropriate independent advice and advocacy support

from local and national sources.

• Provide accurate information to patients, carers and families, about the Trust’s services, and about other health related issues

• Act as a catalyst for change and improvement by providing the Trust with

information and feedback on problems arising and gaps in services.

• Operate within a local network with other PALS in their area and work across organisational boundaries

• Support staff at all levels within the Trust to develop a responsive culture

National Core Standards for PALS National Core Standards are issued by the Department of Health and set out the minimum service to be delivered by PALS. The standards included in this evaluation framework replace those published in 2002 and build on and strengthen the original standards. Please note that the standards have been incorporated into the enclosed evaluation framework. This framework outlines supporting mechanisms and types of evidence that would show that your service is meeting a particular standard. Note that these are best practice examples of actions you could take or evidence you could gather to show you are meeting a standard. They are not requirements. Please use these revised standards forthwith. You are encouraged to use the attached evaluation framework. This will enable you to assess your performance and benchmark against the revised standards and to plan to develop the service you provide in a way that is consistent with other PALS nationally. Please contact your SHA PPI Lead if you have any questions about either the revised standards or evaluation framework 1 Supporting the implementation of Patient Advice and Liaison Services: A resource pack. Department of Health, 2002.

5

PALS is a new and developing service. Therefore these standards will need to be reviewed over time to take account of any changes as the service matures and develops. What is evaluation? Evaluation is about making an assessment – measuring how well a project or activity is doing against a set of criteria. It usually means that those working in the service being evaluated have time to stop and reflect, and to learn from their achievements and disappointments2.

Evaluation means more than assessing whether PALS has complied with its core functions. Evaluation means making a judgement of the project’s effectiveness (i.e. have the original objectives been met) and efficiency (i.e. how well resources are being deployed). It therefore entails looking at outcomes as well as at activities, at relevance as well as numbers, at what could have been done as well as what was done. To evaluate PALS successfully much greater emphasis will need to be placed on qualitative indicators – the invisible aspects of prevention, relationships, trends of referrals, client satisfaction and impact on policies and practice in the service system3.

This evaluation framework This evaluation framework is provided as a resource to help PALS assess their performance against the National Core Standards. It should allow PALS to demonstrate that they are providing a good service and having an impact – and to identify where there are areas for improvement. By sharing a single evaluation framework, there is scope for PALS to compare their performance with others working in similar organisations and with similar resources. But great care needs to be taken in comparing PALS services to ensure that only ‘like is compared with like’. It is important to remember that this framework encompasses an evaluation of the PALS service as a whole. It is not an assessment of individual members of PALS staff – even if the PALS is a single handed service.

2 Evaluator's Handbook (Program Evaluation Kit (2nd Ed.)) Joan L. Herman, Lynn Lyons Morris, Carol Taylor Fitz-Gibbon, 1988. 3 Supporting the implementation of Patient Advice and Liaison Services: A resource pack. Department of Health, 2002.

6

How to use the framework The framework is divided into a number of sections to help you carry out your evaluation. It provides ideas and suggestions as to how you might want to go about this. After this introductory section there are a further six sections which should help you to:

• Plan your evaluation

• Collect information about your local context and the use of your service

• Assess your performance against the PALS National Core Standards

• Assess your performance against other measures e.g. local standards

• Plan how to consult other stakeholders about PALS performance

• Plan and write up your report of your evaluation

It is advisable to assess your performance against all of the National Core Standards, even if there are some that you know you cannot meet. It is important to find out why a standard is not being met and whether this is linked to the service or some other factor. Identifying the barriers to success is crucial to finding ways to improve the service. Examples of previous PALS evaluations are described throughout this framework as well as in the Appendices. These examples provide ideas as to how you can involve other stakeholders in planning and implementing the project and how you might analyse your findings. Further information on methods and contact details for individual PALS are also provided.

7

SECTION 2: PLANNING YOUR EVALUATION The following is a checklist of questions to think about before you start. By being clear about what you want to achieve, you can ensure that the right people are involved all the way through and that the project will meet your needs. Why do you want to evaluate your PALS service? What is the purpose of your evaluation? Be clear about your aims from the beginning. For example, do you want to: • Assess the performance of PALS to improve the service and/or inform staff

development? • Assess the quality of PALS to check whether resources are being used

effectively? • Assess the impact of PALS across a Trust or region?

Ensure that everyone involved in planning the evaluation has discussed and agreed a set of aims. Who will be involved in planning the evaluation? Think about establishing a steering group to oversee the evaluation. This will ensure that the key concerns of various stakeholders are addressed. The group could also help with communicating the results and taking any necessary action. Think about who will make decisions about changes and who will implement them and make sure these people are engaged in the project from the start. Think about whether you want to involve:

• Users of PALS

• PALS staff

• Trust managers

• Members of the Trust Board

• Strategic Health Authority managers

• Community members/patient groups

Who will carry out the evaluation? Do PALS staff have the capacity to carry out an evaluation on top of their normal workload? Is it important that the evaluation is completely independent? What budget do you have for the evaluation?

8

Depending on the resources available any of the following people could carry out the evaluation:

• PALS teams - through self-assessment or via peer-review with other PALS

• External agencies such as university departments

• Other teams within a Trust such as a Quality/Audit team

• Patient groups (in the future the Patients’ Forum)

What will you do with the results? Think about how you are going to use the results of the evaluation. For example, do you want to: • encourage Trust managers and/or PALS to take action to improve the service?

• promote the PALS service to patients, Trust staff and/or the general public?

• share ‘good practice’ with other PALS?

• provide feedback to people who have been consulted as part of the evaluation?

Decide who you want to influence from the start. You can then ensure that you collect the information that you know they will be interested in. Ensure that the relevant stakeholders are prepared to receive the results. When should you carry out an evaluation? Think about timing. If your service is still in the early stages of development, a full-scale evaluation is unlikely to be helpful. You may want to wait until your service has been up and running for at least a year before carrying out an evaluation. However, it may well be useful to collect baseline data early on to help you look at the development of the service. Think about frequency. Your managers, the PCT or Strategic Health Authority may require you to measure performance against standards on a frequent basis (every three or six months). If this is the case, we suggest you use parts A, C and D of the framework in section 3. Small-scale evaluations might be useful once a year to see where immediate improvements can be made to the service. More in-depth and investigative evaluations may only be necessary once every 3-5 years. Think about context. Could the evaluation be used to satisfy any other assessments of the service e.g. CHI reviews? When would a formal report be needed for the Trust Board or the Strategic Health Authority? Think about resources. How long will the evaluation take and what staff time will be required? What budget is available?

9

Summary Record your conclusions here. Why do you want to evaluate your PALS service? Who will be involved in planning the evaluation? Who will carry out the evaluation? What will you do with the results? What is the budget and timescale for your evaluation?

10

SECTION 3: COLLECTING LOCAL INFORMATION Information about the local context will be important for interpreting the results of your evaluation. It will also be essential for meaningful comparisons of PALS to be made – ensuring only like is compared with like. What information should be collected about the local context? Other PALS evaluations have looked at: • When the service began and length of time in operation

• The development of the PALS service, including the resources made available

• Staffing levels

• Type of service provided e.g. telephone or walk-in service

• Accommodation

• Geographical location of the service within organisation e.g. is the PALS office in a hospital reception?

• Type of Trust

• Community served by the Trust

What information should be collected about the use of PALS? Other PALS evaluations have looked at the: • Number of contacts/enquiries on a monthly basis and changes over time

• Source of contacts – patients or Trust staff and trends in referrals to PALS

• Who is using PALS – ethnicity, age, gender etc.

• Types of enquiry - information, liaison, referral or support through complaints

• Source of problems - clinical division, specialty or ward/department

• Signposting to other organisations

• Percentage of enquiries effectively dealt with and resolved

• Contribution of PALS staff to patient and public involvement (PPI) activities

Think about what information you need to collect about your local context and use of the service to meet your specific aims and objectives.

11

How to use the local information you collect There are a number of ways in which you can analyse the information collected about your local situation, depending on the aims of your evaluation. You may want to demonstrate that demand for the service is increasing; you may want to argue for more resources by demonstrating unmet demand; you may want to show support for the service among local stakeholders or to demonstrate that a wide range of the local community is using the service. Some PALS may wish to demonstrate that the service is having an impact on complaints. Examples of how other PALS have used local data are described below. Example 1: University College London Hospitals NHS Trust Analysis of the number of cases handled by PALS on a monthly basis showed a continuous increase in the workload of the service. It also showed that the majority of enquirers were referred to the PALS service by Trust staff. Once staff had used the service, they continued to refer patients on a regular basis. This indicated that Trust staff found the service to be supportive of them as well as of patients. Further details of this evaluation are provided in Appendix 4. Example 2: South West Yorkshire Mental Health NHS Trust Analysis of PALS record of activities was used to describe: • How many contacts were made to PALS on a quarterly basis • Who raised the issue – to identify where there might be gaps in local outreach • What activities were undertaken by PALS in response to enquiries • How many issues were dealt with in full or resolved - as a measure of PALS

achievement Further details of this evaluation are provided in Appendix 2. Example 3: Hull and East Yorkshire Hospitals NHS Trust An analysis of local data was used to show that the service had rapidly expanded in its first few months and that PALS had had a significant impact on the number of formal complaints received by the Trust. The contacts with PALS were analysed by department or specialty and common issues raised by patients were identified e.g. lack of information as to when results can be expected. Further details of this evaluation are provided in the Appendix 6.

12

SECTION 4: ASSESSING PERFORMANCE AGAINST THE NATIONAL CORE STANDARDS FOR PALS How should you complete this assessment against the National Core Standards? You are asked to assess your performance separately under each standard. They are listed one per page. There are then four parts to each assessment: Part A: Outlines the supporting mechanisms and types of evidence that

would show that your service is meeting a particular standard. Note that these are examples of actions you could take or evidence you could gather to show you are meeting a standard. They are not requirements.

You may know of other evidence relating to your performance and you should include this if you wish. Part B: Asks you to consult other stakeholders (e.g. patients and Trust staff) to obtain their views on your performance against the standard. Some suggestions are given as to whom you might consult. You may have other views about who you want to consult locally.

Some suggestions as to how you might consult other stakeholders are provided in the examples of PALS evaluations in the Appendices.

Part C: Asks you to analyse the evidence to assess whether you are meeting the standard. If you are meeting the standard, you are asked to consider what impact this has had. If you are not meeting the standard, you are asked to identify what further action is required. Part D: Asks you to decide on a final rating of your performance against the standard. This rating is based on the traffic light system: Red Urgent attention needed Amber Some work needed Green Doing well The rating scale may help you to think about where to concentrate your efforts to improve the service. However, it is entirely optional. The report and action plan are the most important parts of the evaluation. A final key areas for action section enables you to record the most relevant and important areas for improvement that you have identified as a result of completing the assessment.

13

For a quick evaluation – only complete parts (A), (C) and (D). For a more in depth evaluation involving a range of stakeholders - complete all four parts (A) – (D). In the core section of this framework, all the pages relating to the assessment of your performance against the national core standards are headed in green. Please note these pages are not worksheets. You are not expected to write the information on each page but to record the information elsewhere.

14

STANDARD 1 The PALS service is identifiable and accessible to the community served by the Trust (A) What has been done to meet this standard? Supporting mechanisms

• Needs assessment undertaken to identify effective access methods to meet local needs • Action plan to prioritise and develop access methods • Communication and training strategy developed to promote PALS across the Trust

Types of evidence

• Range of methods of access available e.g. in person, outreach, interpreting and translated materials etc…

• Action plans developed for PALS training and publicity across the Trust • Feedback from service user surveys-strengths and gaps identified for further service

development Additional evidence (optional)

(B) What do other stakeholders think about PALS performance against this standard? For example are Trust staff aware of the services PALS offers? Have users of PALS found it easy to access the service? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

15

(C) Conclusions Has the standard been met? What difference has this made? For example, has PALS been able to reach groups that have not been reached by the Trust before? Has the number of enquires increased following action to promote the service? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating

Urgent attention needed

Some work needed

Doing well

17

STANDARD 2 PALS will be seamless across health and social care (A) What has been done to meet this standard? Supporting mechanisms

• Information sharing agreements in place across and within organisations

• Service procedures shared across organisations

Types of evidence

• Joint PALS/customer care training across health and social care

• PALS accessible from health and social care services/venues

• Feedback from service users - appropriate referrals partnership working between health and social care agencies to resolve service user issues

• Evidence of learning from anonymised service user cases involving health and social

care issues

• Networks developed with PALS/social care across local networks/SHA area Additional evidence (optional)

(B) What do other stakeholders think about PALS performance against this standard? For example, do users of PALS feel they have been signposted or referred to other organisations appropriately? Are staff in other local organisations satisfied with referral protocols? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

18

(C) Conclusions Has the standard been met? What difference has this made? For example, has working with another organisation enabled a patient’s query/concern to be dealt with, when it would not otherwise have been possible? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

19

STANDARD 3 PALS will be sensitive and provide a confidential service that meets individual needs (A) What has been done to meet this standard? Supporting mechanisms

• PALS provides a confidential service

• Information is shared appropriately across and within organisations to resolve individual service user issues

• PALS service procedures take into account the individual needs of service users in the

services it provides and develops Types of evidence

• Confidentiality integral part of job descriptions/PALS service procedures

• Information sharing agreements in place within and across organisations that address confidentiality/Caldicott

• Service user feedback/surveys - evidence of learning/service development from

feedback

• PALS training addresses confidentiality/customer care

• Individual issues resolved within agreed service procedure frameworks or referred appropriately for further action e.g. ICAS, trust, patient forums etc

Additional evidence (optional)

(B) What do other stakeholders think about PALS performance against this standard? For example, what have users of PALS understood about the confidentiality of service? Are users satisfied that the service is confidential? Do Trust staff believe the service to be confidential? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

20

(C) Conclusions Has the standard been met? What difference has this made? For example, have people approached PALS who may not have otherwise voiced their concerns? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

21

STANDARD 4 PALS will have systems that make their findings known as part of routine monitoring in order to facilitate change (A) What has been done to meet this standard? Supporting mechanisms

• Communication and reporting procedures/clinical governance systems in place within and across appropriate organisations to illustrate PALS findings

• PALS undertakes regular impact assessment of its service

Types of evidence

• Outcomes of impact assessment incorporated into regular reporting mechanisms made available to Trust board, patient forums, service users, public, OSC’s, SHA-performance management systems

• Evidence of learning/service development as a result of PALS reporting - resulting

action plans, examples of service change, case studies

• Evidence of feedback given to service users, public-patient prospectus, PCT newsletters/incorporated in PCT communication strategy


(B) What do other stakeholders think about PALS performance against this standard? For example, do members of the Trust Board/Patient Forums believe themselves to be fully informed of the work of PALS? Do PALS staff believe their findings are reaching the right audience? Have changes been made to services or the environment based on PALS work? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

22

(C) Conclusions Has the standard been met? What difference has this made? For example, has there been an impact on the Trust services, culture or environment? Have patients benefited from any changes made? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

23

STANDARD 5 PALS enables people to access information about Trust services and information about health and social care issues (A) What has been done to meet this standard? Supporting mechanisms

• Mechanisms are in place to enable PALS to access current appropriate information from within the trust as well as from other agencies

Types of evidence

• Service user feedback

• PALS data systems and procedures facilitate broad access to a range of health and social care information on behalf of service users


(B) What do other stakeholders think about PALS performance against this standard? You may have obtained feedback from users as part of the evidence collected in part (A). Do users of PALS feel they have been signposted to appropriate and up to date information? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

24

(C) Conclusions Has the standard been met? What difference has this made? For example, have changes been made to the Trust’s patient information or information produced by other organisations? Has out of date information been removed from circulation? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

25

STANDARD 6 PALS plays a key role in bringing about culture change in the NHS, placing patients at the heart of service planning and delivery (A) What has been done to meet this standard? Supporting mechanisms

• Communication and reporting systems in place within and across appropriate organisations to illustrate PALS findings and subsequent changes in service delivery

Types of evidence

• PALS acts as a signpost for people wishing to become more involved in the Trust’s patient and public involvement initiatives

• Evidence of learning/service development actual or planned as a result of PALS

reporting-resulting action plans

• Examples of service change, case studies incorporating service user feedback

• Outcomes/recommendations resulting from trust boards

• Learning incorporated into PALS staff induction training and PALS service development

• Support/training needs of PALS staff identified, plans to address these in place Additional evidence (optional)

(B) What do other stakeholders think about PALS performance against this standard? For example, are Trust staff aware of this function of PALS and have they asked for appropriate PALS input? Have they been satisfied with the support from PALS? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

26

(C) Conclusions Has the standard been met? What difference has this made? For example has the Trust become more patient centred? If so, how? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

27

STANDARD 7 PALS will actively seek the views of service users, carers and the public to ensure effective services (A) What has been done to meet this standard? Supporting mechanisms

• Trust patient and public involvement strategy in place Types of evidence

• PALS service user survey system in place

• PALS procedures incorporate methods to involve patients, carers and the public in service monitoring, planning and development

• Action plans in place/being developed to involve service users in PALS delivery and

development

• Service user feedback used to develop PALS service Additional evidence (optional)

(B) What do other stakeholders think about PALS performance against this standard? For example do users of PALS feel they have been able to influence the way the service has developed? For suggestions on how to consult other stakeholders see Section 6 and the Appendices.

28

(C) Conclusions Has the standard been met? What difference has this made? For example, have there been changes in the way PALS is run, because of input from users of the service? How has this improved the service? Is further action needed to meet this standard? If so, what action is required? By whom? Could the service be improved in this area? If so, how? By whom?

(D) Final rating


Some work needed

Doing well

30

Summary Sheet

Rating of performance against

this standard

Standard

Red Amber Green 1. The PALS service is identifiable and

accessible to the community served by the Trust

2. PALS will be seamless across health and social care

3. PALS will be sensitive and provide a confidential service that meets individual needs

4. PALS will have systems that make their findings known as part of routine monitoring in order to facilitate change

5. PALS enables people to access information about Trust services, and information about health and social care issues

6. PALS plays a key role in bringing about culture change in the NHS placing patients at the heart of service planning and delivery

7. PALS will actively seek the views of service users, carers and the public to ensure effective services

31

KEY AREAS FOR ACTION What three key areas for action have arisen from this assessment? It may help to identify your key areas for action by selecting the standards where the extent to which you meet the standard is low and your ability to improve in this area is high. KEY AREA 1 KEY AREA 2 KEY AREA 3 _____________________________________________

32

SECTION 5: ASSESSING PERFORMANCE AGAINST OTHER MEASURES You may also wish to assess your performance against locally defined aims, principles or standards. For example, some PALS defined local aims and standards in their business plan when they first established the service; others are still developing standards around processes and service delivery. If you are going to assess performance against locally agreed standards, you will need to think through how you will measure your performance against them. Ensure that local stakeholders endorse the measure you are using. Some examples of how PALS have already done this are given below. Example 1: University College London Hospitals NHS Trust The PALS service set itself a standard of giving an initial response to all enquirers within 24 hours. The PALS manager was able to refer to records of contacts to assess whether the service had met this standard. Further details of this evaluation are provided in Appendix 4. Example 2: South West Yorkshire Mental Health NHS Trust Early in the development of its PALS, the Trust consulted key local stakeholders to produce a document called PALS: Values and Standards for Good Practice, which became the philosophy for the service. The principles in this document formed the framework for their subsequent evaluation. Further details of this evaluation are provided in Appendix 2. Example 3: Wakefield West Primary Care NHS Trust The PALS established standards at an early stage to shape the activities of the service. These related to the time taken to respond to a patient, when an enquiry was made in person at a GP’s practice, or at the PALS Headquarters. They detailed how the hub and spoke model would work and the role of district nurses and health visitors in referring patients to PALS. They also defined a procedure for dealing with email inquiries. Performance against these standards formed part of their subsequent evaluation. Further details of this evaluation are provided in Appendix 5.

33

SECTION 6: PLANNING TO CONSULT OTHER STAKEHOLDERS Introduction Many people involved in leading a PALS evaluation will already have experience of choosing the best method to gain information from stakeholders - designing questionnaires, interview schedules and so on. However, if you don’t already have this kind of experience, you may find it useful to read this section. It provides a brief guide to the range of methods you can use to obtain stakeholder feedback. You could also seek advice and help from the Clinical Audit or Clinical Effectiveness team in your Trust. The Trust R&D manager (if you have one) may also offer advice or help. What methods should I use to gain feedback from people? There are a variety of methods you can use to gain feedback from people (whether service users, staff or other stakeholders). The most common of these are:

• Questionnaires – these can be sent by post, carried out over the phone or in person, or distributed through a clinic or ward

• Focus groups • Interviews

Diaries, critical incident technique and patient ‘shadowing’ are also used, but these are probably less appropriate for use in a PALS evaluation. You can read more about these methods on the NHS Modernisation Agency website – see the end of this section for the address and more information. The method or methods you choose will depend on the kind of information you want to gather and the resources you have available (time, skills and money). Each of the most common methods is described briefly below. Questionnaires Questionnaires have been used in many PALS evaluations. They need not take long to complete, they are relatively easy to analyse, and they can generate good quality quantitative data (i.e. data about numbers). However, questionnaires are not always easy to design, so it’s good to get advice from someone with experience. Questionnaires can give you information about the views of a wide range of people, but the amount of information you receive will be fairly narrow, and will be dictated by the questions you ask – so they will not tell you about the aspects of the PALS service that are most important to people, unless you ask this question directly. Most questionnaires contain mainly ‘closed’ questions (that is, questions with a limited range of answers, such as yes or no, or good or poor). ‘Open’ questions, which ask respondents to write about their views, are less frequent. They are harder

34

to analyse and can make a survey time-consuming to complete – this may affect your response rate. So think about using a small number of open questions (for example, ‘do you have any other comments?’) to ensure respondents are able to comment on all the issues that are important to them. If you are sending surveys out by post, it’s helpful to give people an addressed envelope in which to return the survey. Put a stamp on it if the survey is being sent to anyone outside the Trust. Response rates to surveys are typically not high – around 30% is an often quoted figure. Focus groups A focus group typically brings together 8-12 people to talk about a particular subject. Before the focus group, you need to select a list of about six topics, and turn them into questions (for example, how did you find out about the PALS service?). In addition to a facilitator, it is useful to have an observer present. A focus group usually runs for between one and two hours. The main benefits of a focus group are that participants have a chance to share their views, and to comment on each others’ views. They are quicker and cheaper than doing a series of interviews (see below), and they can be set up fairly easily. Focus groups can give you a wide range of information from a small group of people. However, a focus group may be dominated by one or two people, and it tends to be made up of individuals who are available and prepared to give up their time. This means that they may not be suitable if you want to reach a wide range of people. The quality of information gained form a focus group depends largely on the skills of the facilitator, so if you haven’t run a group before, it may be better to approach someone else to do this. Interviews There are three types of interviews – informal interviews, which are precisely that; semi-structured interviews and structured interviews. In semi-structured interviews, the interviewer works out in advance the areas she/he wants to cover, but can vary the order and the wording used. In a structured interview, the interviewer asks fixed questions in a set order – so it’s like using a self-completion questionnaire, but carrying it out in person. Interviews don’t require a lot of preparation. Semi-structured interviews can give you detailed information, enabling you to identify the issues which are important to respondents. But they can be very time consuming to carry out and to analyse. Some researchers use focus groups and/or interviews to find out what issues are important to people, and then design questionnaires to test this on a wider range of people.

35

Who do I need to contact to get ‘representative’ feedback? If you are seeking the views of patients or staff, you need to ensure that, if possible, the people you contact are ‘representative’ of the larger population. For example, if you want to know what nurses in the Trust think of the PALS service, you need to seek the views of nurses at a range of levels and in a range of departments. If you want to know what patients think of the PALS service, you’ll need to think about whether you can ensure that the people you contact reflect the population served by the Trust in terms of age, gender, social class and ethnicity. This is not an easy task, so seek advice, and if you are not able to do this, do not claim that your sample is ‘representative’. Most PALS evaluations have contacted a range of staff and/or people who have used PALS services. They have not sought to have a representative sample, and have been clear about this in their reports. If you are seeking the views of people who have used PALS services, as a minimum try to avoid selecting the people with whom you have regular contact. Getting approval to seek feedback If you intend to seek the views of patients, carers, service users or NHS staff about PALS, you’ll need to obtain ethical approval before you start. Your Local Research Ethics Committee (LREC) can give you this approval – you’ll need to outline the aims of your project, the people whose views you are seeking (e.g. patients, staff etc) and your method (e.g. questionnaire, focus group etc). There is usually an application form for you to complete. LRECs will want to check details about confidentiality, data protection and anonymity, as well as checking that your project will not be harmful to patients, carers or staff. Contact your LREC to get advice about what they need to know from you. Note that obtaining ethical approval can sometimes take a while, so allow plenty of time for this. You also need to keep in mind the requirements of the Data Protection Act. Information you have collected on a database about users of PALS cannot be passed to anyone outside the Trust, and may not be used for purposes other than those which were explained to the PALS user. So you can’t pass names and contact details of PALS users to an external evaluator, for example, unless they have an honorary contract as Trust employees. If you are employing an external evaluator, you may need to send out questionnaires on their behalf. Seek advice from Clinical Audit or IT staff and from the Trust Caldicott Guardian before you set out on an evaluation. You should also ensure that respondents cannot be identified, and that you keep responses separate from contact details.

36

Additional sources of information This section is not meant to be an exhaustive guide to seeking feedback. If you haven’t done this kind of work before, do read up about it. Useful websites include: www.doh.gov.uk/acutesurvey/inpatientsurveys.pdf This document was put together by the Picker Institute Europe to assist people using the National Patient’s Survey. It has good general information on:

• Conducting a postal survey • Resources needed • The advantages and disadvantages of different methods used for gaining

feedback from people • Sample sizes and sampling methods • Data Protection issues • Timescales • Listening to the views of people who do not speak English

This information is of relevance to those working outside acute Trusts as well as those working within them. www.modern.nhs.uk/improvementguides/patients The NHS Modernisation Agency website has short, useful guides on critical incident technique, focus groups, patient shadowing, patient diaries, and discovery interviews. There is also a section on suggested reading. www.corec.org.uk The Central Office for Research Ethics Committees (COREC) website gives contact details for your Local Research Ethics Committee, as well as outlining the kind of projects that need ethical approval, and how to obtain it. www.conres.co.uk Consumers in NHS Research promotes the active involvement of patients, carers, service users and others in research – including evaluations. They have produced guidelines on involving service users in meetings (within their publication Involving Consumers in Research in the NHS: Briefing Notes for Researchers), which could be helpful if you are planning to run focus groups. You can download this publication free of charge from the website. Useful books include Robson, C Small Scale Evaluation London Sage Publications 2000 Krueger, R Focus Groups: A Practical Guide for Applied Research London Sage Publications 1994 Both of these books are available from the King’s Fund Library in London – anyone can use this reference library free of charge, wherever they are based. For more information, see www.kingsfund.org.uk/eLibrary/html/

37

SECTION 7: WRITING YOUR REPORT Your report should be written with your target audience in mind to emphasise the issues that are going to be most relevant to them. As a guide, it might be helpful to structure your report as follows: • an introduction • the aims of the evaluation and why it was commissioned • a brief description of your PALS service – how it is structured, what resources are

available and how it is being used • a section on your performance against national core standards – where standards

are exceeded, where they are being met and where more work needs to be done to meet the standard.

• a summary of your conclusions – emphasising the positive aspects of the

evaluation. You should describe where you think the service is performing well, make recommendations for the three key areas for action to improve the service and identify who needs to take action. Remember it may be just as important to assess where there are barriers to PALS having an impact as to where PALS is making a difference.

• the name of the lead on evaluation and the date of writing the report

38

APPENDICES

PALS’ EXPERIENCES OF EVALUATION

In Appendices 1-6, six examples of previous PALS evaluations are described in more detail. There is a one page summary of each example that outlines why the evaluation was carried out, when it was done, who did it, how they did it and what they found out. Details of the methods used to involve and consult other stakeholders are also provided. If you want to find out more about any of these examples, you can contact the member of staff who was involved. Their name and email address is provided.

39

Appendix 1: Harrogate Health Care NHS (Acute) Trust Why did they evaluate PALS?

To measure the degree of user satisfaction.

When did they do it?

September – October 2002, six months after the PALS service was started.

Who carried out the evaluation?

The Department of Clinical Effectiveness in the Trust.

How did they do it?

Questionnaires were given to service users – 48 out of 75 returned.

What did they conclude? ⇒ PALS was very much appreciated by users of the service.

⇒ PALS fulfilled a need for rapid investigation and response to patients concerns.

⇒ Lack of communication of information to patients was identified as a problem area.

⇒ PALS co-ordinator was praised for her sympathetic listening skills and communication with staff and users.

Contact: Rosemary Petty, PALS Co-ordinator

Email: [email protected] The questionnaire given to service users is over the page.

40

Patient Advice & Liaison Service (PALS) Questionnaire

The PALS service provides confidential, on the spot advice and support, helping you to sort out any concerns you may have about the care we provide, guiding you through the different services available from the NHS. We would

be grateful if you could complete the following questionnaire. Your comments will be used to ensure that the PALS service is responsive to all your needs.

Please contact the PALS Co-ordinator for further information.

______________________________

Please tick the group that represents your age: � 18-30 � 31-44 � 45-65 � 65+ Are you: ❑ The Patient ❑ A Relative or friend of a Patient ❑ Hospital Staff How did you become aware of the PALS service? � The Hospital staff � Local Press � PALS Leaflet � Hospital Display Board � Word of Mouth � Other (Please state) __________________________________ How did you contact the PALS Co-ordinator? � By Telephone � By Referral Slip � In Person Did you find it easy to contact the PALS Co-ordinator? ❑ Yes ❑ No How long did it take for your issues / concerns to be resolved by the PALS Co-ordinator? ❑ Same Day ❑ 1-3 Days ❑ 3-7 days ❑ 7+ days ❑ Not sure

Please Comment

Harrogate Health Care NHS (Acute) Trust

41

How re-assured did you feel after your discussions with the PALS Co-ordinator? � Very re-assured � Fairly Re-assured � Not re-assured � Not sure Did you feel that the PALS Co-ordinator listened to your concerns and issues? ❑ Yes ❑ No ❑ Not sure How easy to understand was the explanation given to you by the PALS Co-ordinator? ❑ Very easy to understand ❑ Fairly easy to understand ❑ Some parts easy/some not ❑ Not sure Would you use the PALS service for any future issues / concerns? � Definitely � Maybe � Not sure

Please Comment

Please Comment

Please Comment

42

If the PALS service did not exist, would you have made a complaint? ❑ Yes ❑ No ❑ Not sure Please provide any further comments.

Thank Thank you.

Please Comment

43

Appendix 2: South West Yorkshire Mental Health NHS Trust

4http://cbe.unn.ac.uk/PALS/intro.htm

Why did they evaluate PALS?

To review the pathfinder PALS established in May 2001, one year after the service began.


May – July 2002


The PALS Project Board commissioned the Quality, Evaluation and Development Department of the Trust to review the service for service users, carers and staff. The Project Board consists of a wide range of local stakeholders who meet every two months and provide overall steering, direction and accountability for PALS.

How did they do it?

The evaluation was based on work carried out by the University of Northumbria at Newcastle4 as well as a set of locally agreed values and standards. It involved reviewing data from: • a report on the progress and development of the service written by

the PALS manager • the database of PALS enquiries • a questionnaire survey of service users – 84 out of 465 returned • a focus group with 12 ex- and current service users and carers • semi-structured interviews with the PALS team • a questionnaire survey of the PALS Project Board – 4 out of 17

returned • a questionnaire survey of Trust staff – 50 out of 200 returned

What did they conclude? ⇒ The service was widely known and viewed positively by everyone

who had heard of it. ⇒ Service users, clients and carers thought PALS was a good service

and had made a difference. ⇒ The team felt that they had provided a flexible and dynamic service

that would respond to change and could continue to improve over time. They agreed outreach was the main area to concentrate on in the future.

⇒ There was no clear view of how PALS had made an impact on other Trust services, but it was felt it had the potential to do so, as the service became more entrenched in Trust structures.

Contact: Phil Walters

Email: [email protected]

44

Patient Advice and Liaison Service (PALS)

Service Users’ and Carers’ Feedback Survey The Patient Advice and Liaison Service (PALS), is a service for people who use or come into contact with the mental health and learning disability services provided in Wakefield and Pontefract areas. Our PALS is based at Fieldhead, and has been running for almost one year.

Please take a few minutes to give us YOUR views on PALS. We are committed to responding to your views so that we can improve our PALS, and help shape new PALS in other health services. All the information you give us will be anonymous. Thank you. Please tick ONE box only unless specified

1. Are you a service user or client of mental health/learning disability services in Wakefield and Pontefract, or a carer?

Service user / client

Carer (a carer is a person who gives a considerable amount of help to a relative or friend who is ill, disabled, elderly or who has a mental health/learning disability problem and could not cope without their help).

2. Did you know about PALS at Fieldhead?

(Please remember that the shop and café are NOT PALS)

Yes No Please go to Q11 3. How did you hear about PALS? (Please tick ALL that apply)

Word of mouth Saw a poster

Saw a leaflet Member of staff told me

Other ways (Please tell us below)

………………………………………………………………….……………………………………………………………………………………………………………………

Please turn over

45

4. If you have used PALS, please tell us how (Please tick ALL that apply)

I haven’t used PALS (Please go to Q11)

I have ‘dropped in’ to the PALS office at Fieldhead

I have phoned PALS

I have picked up leaflets and written information

Other ways (Please tell us below)

………………………………………………………………….………………………………………………………………………………………. 5. What has been the main reason that you have used PALS? (Please tick one)

For information e.g. leaflets, books, local support groups

To gain access to the Richmond Fellowship Advocacy Service

For general health information

For help with how to make a complaint

For help to resolve a concern about your care

Another reason (Please tell us below)

………………………………………………………………….……………………………………………………………………………………….

Or would you rather not say? 6. Are you generally happy with the way PALS has helped you?

Yes No

7. Would you recommend PALS to other service users or carers that you know?

Yes No 8. Can you think of up to 3 things about PALS that you liked?

1.

2.

3.

……………………………………………………………….……………………………………………………………………………………….

……………………………………………………………….……………………………………………………………………………………….

……………………………………………………………….……………………………………………………………………………………….

9. Can you think of up to 3 things about PALS that you would like to see changed?

1.

2.

3.

……………………………………………………………….……………………………………………………………………………………….

……………………………………………………………….……………………………………………………………………………………….

……………………………………………………………….……………………………………………………………………………………….

46

10. Based on your experience of PALS, please give your opinion of the following statements about PALS. Strongly

Disagree Disagree

Neither disagree nor agree

Agree Strongly Agree

Don’t know

Anyone can use PALS

PALS staff support service users, clients and carers to speak for themselves and do not impose their own views

PALS is quite an informal service

PALS respects service users’, clients’ and carers’ privacy and dignity

PALS does not make judgements about service users, clients and carers

PALS tries to resolve the issues raised by service users, clients and carers quickly and efficiently

PALS gives feedback when promised

PALS is a confidential service

PALS only takes an issue forward when the service user, client or carer wants them to

PALS helps service users, clients and carers to get involved in groups or projects to improve mental health/ learning disability services

PALS helps service users, clients and carers to get involved in mental health and learning disability-related groups or projects

11. Please add any further comments you would like to make about PALS.

………………………………………………………………….……………………………………………………………………………………………………..

……………………………………………………………………………………………………………………………………………………………………..

Thank you for giving us your views. Please seal this questionnaire in the envelope provided and give to …………(name of linkworker)OR post to the QED department, Fieldhead in the pre-paid envelope provided, by 14th June 2002. (you do not need a stamp).

47

South West Yorkshire Mental Health NHS Trust Patient Advice and Liaison Service

Evaluation of PALS

PALS Focus Group Proforma

Groups: Service Users’ Forum, Carers’ Mental Health Advisory Group Introductions and brief summary of purpose of focus group.

1. Please tell me how long this group has been going… and which came first – the

PALS service or this group (SUF/CMAG)?

2. What sort of influence has your group had on the way PALS has been

developed?

Prompts – were the ideas for PALS discussed here? Were your suggestions incorporated in to PALS as it was set up? 3. What are the links between your group and PALS?

Prompts – who sits on both groups? How do you find out what happens with the development of the PALS service?

4. What are people’s impressions of the way PALS works with users and carers?

Prompts - Good/bad/OK?

5. How much do you think PALS has an impact on the Trust, in terms of

Giving users and carers a voice?

Helping users and carers to access better care?

Helping users and carers to access better information about mental health services?

Facilitating the complaints process?

Increasing awareness amongst staff of the importance of users’ and carers’ views?

6. Do you think on the whole PALS meets the needs and demands of the users and

carers?

Prompts - Why and how?

7. What would you want changed about PALS in the future?

Thank you and brief description of what happens next, data anonymity etc

48

South West Yorkshire Mental Health NHS Trust Patient Advice and Liaison Service

Evaluation of PALS

PALS Interview Proforma

About the development of PALS and the PALS Board 1. What do you think about the overall standard of leadership and direction of the

PALS project 2. Please give me an example of the important things that PALS leadership does

which has contributed to PALS development 3. What do you consider to be the most important function of the PALS Board? 4. Do you think the PALS Board has achieved this function?

(scale from… never - all the time) 5. Do you think the Board has appropriate representation?

(Comments) 6. Do you think the PALS Boards has the appropriate links with other strategic

elements of Wakefield and Pontefract Community Health NHS Trust management – such as the Mental Health Service Board, Trust Board? Comments

7. What changes could be made to the PALS Board to further improve its role? PALS Standards and philosophy 8. Do you feel PALS is loyal to its values and standards for good practice

(Philosophy statement) - show copy to refresh memory 9. Is easy or difficult to follow this philosophy? 10. What are the difficult areas? 11. How might these be resolved? What impact has PALS had? 12. In your opinion, has PALS had an impact on the Trust, in terms of

Giving users and carers a voice? Helping users and carers to access better care? Helping users and carers to access better information about mental health

services? Facilitating the complaints process? Increasing awareness amongst staff of the importance of users’ and carers’

views?

49

PALS processes 13. Is PALS flexible to change? Any examples of this? 14. What is communication like between PALS and the rest of the Trust? 15. Do you think PALS is a fast or slow service? Why? Users and carers who access PALS 16. Do you think you meet the needs and demands of the users and carers who

access PALS? 17. Why and how? 18. Please give me up to three

Good things about PALS as it is now?

Areas for development and change for PALS in the future Thank you for your time. The information you give me will not be attributable to you as an individual but will be grouped as the responses of the PALS team.

50


PALS Board questionnaire

The Patient Advice and Liaison Service (PALS), for the whole of the area and services provided by the former Wakefield and Pontefract Community Health NHS Trust, has been in operation at Fieldhead for almost one year. The PALS service is located next to the shop and café at Fieldhead. PALS is a service for people who use or come into contact with the mental health and learning disability services. We are committed to responding to service users’, carers’ and staff views. The information we get from your views will help to improve our PALS and shape PALS across the new Trust, and PALS in other health services. As a member of the PALS Project Board, which has had a key role in the development of PALS, we would like to ask you to take a few minutes to give us your views on PALS. All the information you give us will be treated in confidence and will be anonymous. We have attached the Terms of Reference of the PALS Project Board to the end of the questionnaire as an aide-mémoire. Please tick ONE box only unless specified SECTION 1: ABOUT THE DEVELOPMENT OF PALS

1. Do you think the overall standard of leadership and direction of the PALS project was:

Excellent Good Average Below Average Poor 2. Please give an example of an important way the leadership for PALS has contributed to PALS

development:

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..……………. …………………………………………………………………………………………………………………………

3. What do you consider to be the most important function of the PALS Board?

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..……………. …………………………………………………………………………………………………………………………

4. Do you think the PALS Board has achieved this function?

Never Occasionally Nearly Always All the time

51

5. Do you think the Board has appropriate representation?

Yes No

Please comment

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..……………. …………………………………………………………………………………………………………………………

6. Do you think the PALS Board has the appropriate links with other strategic elements of

Wakefield and Pontefract Community Health NHS Trust management, e.g. the Mental Health Service Board and the Trust Board?

Yes No

Please comment

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..……………. …………………………………………………………………………………………………………………………

SECTION 2: ABOUT THE FUNCTION OF THE BOARD IN THE FUTURE

7.

What changes could be made to the PALS Board to further improve its role, e.g. more meetings, less meetings, different representation, other suggestions? .

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..……………. ………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………

SECTION 3: ABOUT PALS IN GENERAL 8. In your opinion, has PALS had an impact on the Trust, in terms of giving service users and carers

a voice?

Yes No 9. In your opinion, has PALS had an impact on the Trust, in terms of helping service users and

carers to raise concerns they may have with Trust services?

Yes No 10. In your opinion, has PALS had an impact on the Trust, in terms of helping service users and

carers to access better information about mental health and learning disability services?

Yes No

52

11. In your opinion, has PALS had an impact on the Trust, in terms of helping individuals access the

complaints process?

Yes No 12. In your opinion, has PALS had an impact on the Trust, in terms of increasing awareness amongst

staff of the importance of service users’ and carers’ views?

Yes No 13. Please list up to 3 good things about PALS as it is now? a.

…………………………………………………………………………………………………………………………………….

b.

…………………………………………………………………………………………………………………………………….

c. …………………………………………………………………………………………………………………………………….

14. Please list up to 3 areas for development for PALS in the future? a.

…………………………………………………………………………………………………………………………………….

b.

…………………………………………………………………………………………………………………………………….

c. …………………………………………………………………………………………………………………………………….

15. Thinking about the original plans to develop PALS, how would you rate PALS’ achievements

against the Board’s expectations?

Fully achieved Partly achieved Not achieved

Thank you for taking the time to complete this questionnaire

Please return it – in the internal post – to the QED department, Fieldhead

by 31 May 2002

53


Staff Survey

The Patient Advice and Liaison Service (PALS) for the whole of the mental health and learning disabilities services in Wakefield and Pontefract, and has been in operation at Fieldhead for almost one year. The PALS service is located next to the shop and café at Fieldhead.

PALS is a service for people who use or come into contact with the mental health and learning disability services. We are committed to responding to service users’, clients’, carers’ and staff views. The information we get from your views, as a member of staff, will help PALS in Wakefield and Pontefract, across the localities of the new Trust and in other health services. Please take a few minutes to give us your views on PALS. All the information you give us will be treated anonymously. Thank you. Please tick ONE box only unless specified

1. Did you know about the Patient Advice and Liaison Service? (Please remember that the shop and café are NOT PALS)

Yes No (Please go to Q7) 2. Have you referred anyone to PALS?

Yes No 3. If you have accessed PALS on behalf of a service user, client or carer, please tell us how?

(Please tick ALL that apply)

I have ‘dropped in’ to the PALS office at Fieldhead

I have phoned PALS

I have picked up leaflets and written information for a service user/client or carer

Other ways (please tell us below)

………………………………………………………………….………………………………………. I haven’t used PALS yet (Please go to Q7)

4. How would it be easier or better for you to be able to contact PALS, to get information or help for

a service user/client/carer/yourself?

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..…………….

54

5. Based on your experience of contact with PALS, please give your opinion of the following statements.

Strongly Disagree Disagree

Neither disagree

nor agree Agree Strongly

Agree Don’t know

PALS does not make judgements about service users, clients and carers

Anyone can use PALS

PALS respects service users’, clients’ and carers’ privacy and dignity

PALS supports service users, clients and carers to speak for themselves

PALS staff do not impose their own views on service users, clients and carers

PALS tries to resolve the issues raised by service users, clients and carers quickly and efficiently

PALS is quite an informal service

PALS gives feedback when promised

PALS only takes an issue forward when the service user, client or carer wants them to

PALS is a confidential service

PALS helps service users, clients and carers to get involved in mental health and learning disability-related groups or projects

6. Would you recommend PALS to service users, clients or carers that you know?

Yes No 7. Please add any further comments you would like to make about PALS.

………………………………………………………………….…………………………………………………….. …………………………………………………………………………………………………………..…………….

Thank you for taking the time to complete this questionnaire Please return it to the QED department, Fieldhead (in the internal post)

by 14th June 2002

55

Appendix 3: London PALS pathfinder sites


To review the experiences of the 12 PALS pathfinder sites in London to inform guidance for the development and management of new PALS.


Six months after the PALS pathfinders had been set up. Evaluation carried out between August and October 2001.


SILKAP Consultants - commissioned by the London Regional Office of the Department of Health. Patient Advocacy and Liaison Services - A Review of the London PALS Pathfinder sites. SILKAP Consultants, October 2001. NHS London Regional Office.

How did they do it?

Rapid appraisal method involving: • Visits to each site • In-depth interviews with key staff • In-depth interviews with users • Site observations

What did they conclude? ⇒ Trusts were listening to feedback and trying to handle patients’

concerns ⇒ There was a lot of enthusiasm for PALS on the ground reflecting

the commitment and calibre of the staff ⇒ It seemed critical that services would have to challenge the status

quo to ensure they did reach vulnerable and disadvantaged groups. This had implications for resources and staff training.

The report made a number of recommendations for: ⇒ Training PALS staff ⇒ Management & structural links ⇒ Publicity and awareness raising ⇒ Data collection & monitoring ⇒ Evaluation ⇒ Improving local service delivery

Contact: SILKAP consultants Email: [email protected]

56

Silkap Review Of The London Pathfinder PALS Interview Schedule for London Pathfinder PALS

The aim is to elicit feedback about the actual experiences, lessons learnt and future priorities for the service. The Schedule is structured to cover the following areas:

Nature and Organisation Of The Service: Establishment / Development Of The Service: Staff Recruitment, Selection And Training Delivery Management Of Service Data Collection Communication Links And Relationships Service Monitoring And Evaluation Service Impact And Outcomes Service Development And Improvement

Please start by completing the respondents’ details below: 1. Organisation and Contact Details Name:

Organisation / Group:

Job Title:

Address:

Post

Code:

Telephone: Fax No:

Email:

Nature and Organisation Of The Service: Rationale

1. PALS Context: (prompts) Type of NHS Trust Population served - how identified

57

Location of service Needs being addressed by service

2. Description of the service provided: (prompts)

Aims Objectives Principles Nos. staff Activities Availability of the services – access times - how was this identified? e.g. weekends, peak periods Access – methods

3. What is the organisational structure of the service? 4. How does the PALS differ from the Trust’s Complaints and Customer Services

Departments?

5. What boundaries are there to distinguish the PALS from other services?

Establishment / Development Of The Service:

6. How the service was established? (prompts) Was it established from nothing? Was it building on an existing service / framework / existing plans

7. Who was involved in developing the service? (prompts)

How were staff involved? How were users / patients involved? How were local communities involved?

8. What was the initial cost of setting up the service?

Staff Recruitment, Selection And Training

9. How many staff are in the service? (prompts) Staff role / jobs / designation / tenure Grades

10. What are their roles / responsibilities / grades?

11. What skills and qualities were identified for PALS staff?

12. How were job descriptions developed?

58

13. How were staff recruited?

14. How many staff have left the service since its establishment?

15. Does the service have its full complement of staff?

16. If not, where do gaps exist?

17. How are recruitment issues being address / resolved?

18. Have you undertaken any training for staff ? If yes, please describe. If no, what training would you like to undertake ?

19. Have you identified any training needs for staff ?

Delivery

20. What is the patient Pathway through the PALS?

21. How are enquires from users / patients dealt with? (prompts) Taking action Providing feedback to users / the Trust

22. What provisions are there to meet the needs of specific user groups, e.g.?

(prompts) People who do not speak English People with hearing difficulties People with sight difficulties People with mobility difficulties People with learning difficulties disabilities People with Mental health Problems Older people Young people Children Women Other groups

23. Does the service undertake any pro-active work that is not initiated by contact

with users / patients?

24. How long do you normally spend on an enquiry (from initial contact to resolution)?

59

Management Of Service

25. How is the service managed?

26. Who manages the services?

27. What mechanisms or structures exist for? (prompts) Managing individual members of staff (including staff support) Staff feedback and discussion about the service Problems solving and sharing information

Data Collection

28. What procedures or systems are in place for recording and monitoring enquires / cases?

29. What, if any, links are there to complaints monitoring systems / data?

30. What data is included in your routine monitoring of the service?

31. What are the common themes for assistance or support that the service

receives?

32. What is the user profile of the service?

33. What IT / Computer software package/s are used? (prompts) Datex AMS Other

34. Are these systems adequate for the monitoring you require / are expected to

carry out?

35. What are the limitations of the system being used? Communication

36. How are users / patients informed about the service?

37. What information is provided about the service for? (prompts) Users / patients / community & voluntary groups Members of staff

38. What information outlets are used? (prompts)

Internally externally

60

39. What methods / activities have been used to raise awareness about the

service? (prompts) Leaflets Posters Presentations Road show Open Days Other

40. What mechanisms are there for on-going communication and feedback?

(prompts) Of issues to the Trust To service users To staff

41. Is there an expectation that the PALS provide feedback to the Trust on

common issues emerging?

Links And Relationships

42. What links does the service have with departments across the Trust? (prompts) Specifically links with Chief Exec. Formal Informal

For example, what links exist to other relevant departments? (prompts)

Complaints Customer services Clinical Governance / Quality Assurance Performance management Other

43. What mechanisms are in place for liaison with related support services, e.g.?

(prompts) Interpreting services Independent advocacy organisations Community groups CHCs CABs Other

44. What is the experience of working with / liasing with other agencies?

61

Service Monitoring And Evaluation

45. How effective is the service at dealing with the problems / issues brought by users / patients?

46. How does the service deal with enquiries / cases which are which are

‘inappropriate’? (prompts) signposting to other ‘appropriate’ services referring to other ‘appropriate’ services providing information about other services

47. Is there a formal internal monitoring and reporting mechanism?

48. Have any systems / methods been developed for measurement and / or

evaluation of the service?

49. What mechanisms are in place to manage unresolved issues?

50. How are issues / enquiries managed which the service is unable to deal with ?

51. What problems / issues have arisen?

Service Impact And Outcomes

52. What are the expected benefits of providing this service from the perspective of? (prompts) Users/patients Staff

53. What are the perceived problems of providing this service from the

perspective of? (prompts) Users/patients Staff

54. How has the actual experience of the service measured against the service

objectives? 55. What impact has the service had on the wider Trust services?

56. What tensions are there for delivering a patient centred service in relation to

the Trust’s own agenda?

57. How is the issue of conflict of interest resolved / dealt with by the service?

58. What impact has the service had on complaints? (prompts) Numbers Type

62

59. How has the service been influenced by: (prompts) users others

60. What issues have been raised by the service which need follow-up action by

the Trust?

61. How has the service influenced the wider agenda for user involvement in the Trust?

62. Do you have any evidence that staff attitudes have been influenced by the

service?

63. Is the emphasis of dealing with user / patients concerns now broader than dealing with complaints?

64. What are the common failures of the service? (additional prompt / catch all

from Q54.)

Service Development And Improvement

65. What areas of the service do you think need to be improved or developed?

66. What constraints have there been to developing the services?

67. Has the level of support within the Trust enabled the service to develop as expected?

68. In what areas do you think greater support is required? (prompts)

Management Training Recruitment User involvement Linkages / liaison across Trust Follow-through Other

69. Where should this support come from? (prompts)

Trust LRO Centre

70. What lessons / issues have been identified since establishment of the service?

71. How have the issues been dealt with?

72. How have the lessons been applied within the service?

63

73. What are the key messages / lessons that you would want to pass on to any new PALS being developed?

74. What changes have you made to the service?

64

Silkap Review Of The London Pathfinder PALS Interview Schedule for Users

The Schedule is structured to cover, the user’s experience of using the PALS. The aim is to elicit feedback about the actual experiences and views about changes or improvements that could be made. General

1. Name of PALS:

2. Date of interview:

3. Time of Interview:

4. Interviewers Name: Using PALS

5. Why did you contact the PALS?

6. How did you find out about the PALS?

7. How many times have you used the PALS?

8. What did you expect to get form the PALS?

9. Did the assistance you received meet your expectations? / Did the PALS help to resolve your enquiry / problem?

10. Were you happy with the way you were treated?

11. Were you happy with the way your enquiry was handled?

12. How long did it take to meet resolve your problem / deal with your concern?

13. Would you use the services again for a similar problem?

14. Would you recommend someone else to use the PALS?

15. If you had not used the PALS where would you have received assistance?

16. Have the PALS staff ever asked for you views about the service? (prompts)

When How

65

17. Have you been asked to participate in the development of the service in any other way? (prompts) Participate in a user group Survey Steering group Advisory group User panel Other

18. What do you most like about the PALS? (prompts)

Its location How your enquiry/ issue/s is handled Staff attitude towards you How quickly your concern is dealt with Other

19. What do you least like about the PALS? (prompts)

Its location How your enquiry/ issue/s is handled Staff attitude towards you How quickly your concern is dealt with Other

20. Is there anything about the services you would like to see changed or

improved? (prompts) Location How long you have to wait to get your concern dealt with The way you were treated? The way your enquiry / concern was handled

66

Appendix 4: University College London Hospitals NHS Trust


UCLH was one of the 12 PALS pathfinder sites. Review of the service formed part of the transition from pathfinder status to fully established PALS. The Trust’s Clinical Governance Committee also requested a review of the service to provide information to inform future decisions about the development of PALS.


January – March 2002. Service established in September 2001.


Led by PALS Manager. Overseen by the Patient Quality and Environment Board (a sub-committee of the Trust Clinical Governance Committee). The Camden User Involvement Network (CUIN) also reviewed the work. CUIN was formed from the Camden CHC User Involvement Working Group and has become a shadow Commission for Patient and Public Involvement in Health.

How did they do it?

• Assessed their performance against national standards • Assessed their performance against UCLH PALS delivery

standards • Carried out client satisfaction survey – 31 out 121 returned. • Carried out a cost comparison between PALS and formal

complaints

What did they conclude? ⇒ UCLH was achieving and far exceeded the required level 1 of the

eight national standards set out in the PALS Resource Pack. In many cases it was achieving the highest level of 5.

⇒ Over 80% of patients were happy with the way they were treated and 67% were happy with the way their query was handled.

⇒ PALS proved less resource intensive than the complaints process. ⇒ Six service improvements had been made during the evaluation

period (from repainting of waiting areas to provision of volunteers in the minor trauma area) and a further nine were identified.

Contact: Sue Benn, PALS Manager

Email: [email protected]

67

University College Hospital PALS Evaluation July 2002 A review of complex PALS cases

A comparative analysis of complex PALS cases with the NHS complaints procedure. The national complaint procedure stipulates what part of the complaints management process is carried out by particular grades of staff. The order follows the normal route of a formal complaint through the national procedure. One session is equal to 15 minutes. An assessment of the time and resources taken to manage a formal complaint. Step 1 Complaints Officer

Receives complaint, entry onto database, Letter of acknowledgement sent to complainant

(2 sessions) Step 2 Complaints Manager Forwarding of complaint to relevant general manager

(1 session) Step 3 Operational Manager

Investigation of complaint, letters or reply drafted (x sessions) where x

should be equal to the number of sessions PALS would take to investigate the same complaint.

Step 4 General Manager Checking draft investigation report (1 session) Step 5 Complaints Manager Final quality check (2 sessions) Step 6 Chief Executive Signing off final response (1 session) Grades of Trust Staff Complaints Officer 8 = PALS Facilitator & PALS Administrator Complaints Manager 10 = PALS Manager Operational Manager 12 General Manager 15 Chief Executive 25 Cost of formal complaint Step 1 2 sessions @ 8 = 16 Step 2 1 session @ 10 = 10

68

Step 3 x sessions @ 12 = 12x Step 4 1 session @ 15 = 15 Step 5 2 sessions @ 10 = 20 Step 6 1 session @ 25 = 25 Total 7 + x sessions = 86 +12x The PALS Process Step 1 PALS Administrator

Receives complaint, entry into database, letter sent/ telephone contact to acknowledge complaint. (2 sessions)

Step 2 PALS Facilitator Investigation of complaint, letters or reply drafted.

(x sessions plus 1 session) where x is the number of sessions PALS took to investigate the complaint, the extra session is taken up with the PALS Manager in the final check.

Step 3 PALS Manager Final check and signing off (1 session) Cost of PALS Step 1 2 sessions @ 8 = 16 Step 2 x sessions @ 8 = 8x Step 3 1 session @ 8 = 8 (PALS Facilitator) 1 session @ 10 = 10 (PALS Manager) Total 4 + x sessions = 34 + 8x Cost of PALS 4 + x sessions = 34 +8x Cost of Formal Complaint 7 + x sessions = 86 +12x Table 1 Cost comparison between PALS and the formal complaints procedure

Case Number

X Cost of PALS

Cost of Formal Complaint

Difference in Cost

1 4 66 134 68 2 27 250 410 160 3 3 58 122 64 4 3 + 98 162 64

69

4(10) 5 23 218 362 144 6 2 50 110 60 7 4 66 134 68 8 7 90 170 80 9 4 66 134 68 10 3 58 122 64

Case 4 required 4 extra sessions of the PALS Managers time, hence 4 sessions valued at 10 is added to the x value in case 4.

Mean Cost of PALS: 102 Mean Cost of Formal Complaint: 186

Mean Difference in Cost: 84 = 45.16% These figures indicate that the Patient Advisory and Liaison Service is 45.16% more cost effective than the formal complaints procedure. Solving complaints via PALS is more economic for the Trust. The process PALS employs involves a three step process, compared to a six step process for formal complaints, and does not require Chief Executive involvement. In conclusion, PALS is a less time consuming process and thus complaints can be dealt with in a more timely and less expensive way.

Figure 5Cost Comparison Between PALS and the Formal Complaints Procedure

0

50

100

150

200

250

300

350

400

450

1 2 3 4 5 6 7 8 9 10

Case Number

Cost of PALS

Cost of FormalComplaint

70

Appendix 5: Wakefield West Primary Care NHS Trust


To identify what was working well and areas for improvement and development.


Six months after the service began. The PALS was set up as a pathfinder site in October 2001.


The Quality Evaluation and Development Department at the South West Yorkshire Mental Health NHS Trust.

How did they do it?

To evaluate PALS from different perspectives the following data was reviewed: • a report from the PALS manager • the record of PALS issues or enquiries • a questionnaire survey of Health Visitors and District Nurses -14

out of 31 returned • telephone interviews with Practice Managers – 8 out of 10 agreed

to be interviewed • a questionnaire survey of patients using primary care services –

105 out of 360 returned • a questionnaire survey of patients who had contact with their

Health Visitor or District Nurse – 27 out of 160 returned • a questionnaire survey of PALS service users – 3 out of 10

returned

What did they conclude? ⇒ A hub and spoke model had been successfully established

⇒ PALS had successfully developed links with other PALS and organisations

⇒ Community nurses proved key to PALS and the majority were referring patients to the service

⇒ Successful training of staff had been delivered – more was called for by practice staff

⇒ Overall awareness of PALS was low and needed to be raised

Contact: Joanne Hinchcliffe, Public Involvement Co-ordinator Email: [email protected]

71

Wakefield West Primary Care Trust Public Involvement Questionnaire


The Patient Advice and Liaison Service (PALS), is a new service for people who use or come into contact with local health services. PALS is there to provide information on a wide range of health and related issues; give contact details of support and voluntary organisations; and to resolve concerns quickly and efficiently.

Wakefield West Primary Care Trust PALS has been in operation for almost six months through GP surgeries and health centres; individual staff such as district nurses/health visitors; and via a freephone helpline.

It is a service for you and we are committed to responding to your views.

We would be grateful if you could take a few minutes to give us your views on PALS. All the information will remain anonymous and will not restrict anyone’s treatment, care, or access to services.

The information we get from your response to these questions will help to develop and improve PALS in Wakefield.

Thank you for your time.

Please tick ONE box unless otherwise stated

Q1 Were you aware of the Patient Advice and Liaison Service?

Yes (Please go to Q2) No (Please go to Q6)

Q2 How did you find out about PALS?

Saw a poster Picked up a leaflet My GP told me

Health Visitor told me Nurse told me Other

(please specify below)

Q3 Have you used PALS?

Yes (Please go to Q3a) No (Please go to Q6)

72

Q3a If you answered ‘yes’ to Q3, what did you use PALS for?

Information about health care services

For contact details of support/voluntary organisations

For information about how to make a complaint

Other (please specify below)

Q3b Where was your contact with PALS? (Please tick all that apply)

At my doctor’s surgery From my Health Visitor

From my District Nurse At Trust Headquarters (Thornhill St)

Other (please specify)

73

Q4 Please give your opinion/impression on the following statements.

Strongly

disagree

Disagree Neither agree nor disagree

Agree

Strongly agree

1 2 3 4 5

It is easy to access the service

A good range of information is available

The staff are friendly and courteous

I received a prompt response

The outcome was satisfactory

Please comment:

Q5 Would you use PALS again?

Yes No Not sure Not applicable

Q6 Would you use PALS in the future?

Yes (Please go to Q6a) No (Please go to Q7)

74

Q6a If you answered ‘yes’ to Q6, what would you use PALS for?

To gain information about health care services

To gain information about support/voluntary organisations

To help resolve concerns about local health services

To gain information about how to make a complaint

Other (please specify below)

Q7 What other services would you like PALS to provide?

Q8 Would you recommend PALS to other people?

Yes No Not sure Not applicable Please make any further comments below


Please return it to the QED department, Fieldhead in the pre-paid envelope provided by 28th March 2002.

75

If you would be interested in feedback from the questionnaire on completion of the evaluation, please fill in your details below and return this sheet with the questionnaire. This sheet will be detached from the questionnaire, so that your views remain anonymous. Please send me a copy of the findings from the Wakefield West PALS Your name: ………………………………………………………………………………… Your address: ……………………………………………………………………………… ……………………………………………………………………………… ……………………………………………………………………………… Telephone number: …………………………………………………………………………

76

Wakefield West Primary Care Trust

District Nurse and Health Visitor Questionnaire

Q1 Do you carry the PALS cards?

Yes No

Q2 Do you carry the PALS leaflets?

Yes No

Q3 Do you carry the recording forms for PALS enquiries?

Yes No

77

Q4 Please give your opinion on the following statements:

Strongly

agree

Agree Neither agree nor disagree

Disagree

Strongly disagree

a) I am able to give patients sufficient

information about accessing PALS

b) I think PALS is able to help patients

with their enquiries

c) I think PALS is easily accessible to

patients

d) I refer patients to PALS when they

have an enquiry

Q5 Do you have any comments you would like to make about any of the above statements?

Q6 Are there any additional ways you think patients should be able to access PALS?

Q7 Do you have any suggestions on how PALS can be developed or improved?


Please return it to the QED department, Fieldhead by 28th March 2002 in the enclosed pre-paid envelope.

78

WAKEFIELD WEST PRIMARY CARE TRUST

PATIENT ADVICE AND LIAISON SERVICE (PALS)

TELEPHONE SURVEY – PRACTICE MANAGERS

SCORE 1 = low, 5 = high 1. Would you feel able to give patients sufficient information about accessing PALS?

Please give a score 1-5 2. Do you think PALS is easily accessible for patients?

Please give a score 1-5 3. Would you refer a patient to PALS if they had a query that you were unable to resolve?

Yes/No Can you give any reasons for your answer?

4. Do you feel that PALS will empower front-line staff to deal with queries?

Yes/No Can you give any reasons for your answer?

5. Are there any additional ways you think patients should be able to access PALS?

Any comments? 6. Do you have any suggestions on how PALS can be developed or improved?

Any comments?

79

Appendix 6: Hull and East Yorkshire Hospitals NHS Trust


To examine progress against the PALS Core standards issued by the Department of Health in January 2002 (in the PALS Resource Pack).


June 2002. The PALS began as a telephone service in April 2001 and the first full-time PALS officer came into post in August 2001.


Clinical Audit Department, Hull & East Yorkshire Hospitals NHS Trust

How did they do it?

The evaluation relied on four main source of information: • PALS activity data • Patient views – semi-structured telephone interviews used to set

the questions for a postal survey • Staff questionnaire • Informally obtaining the views of partner organisations – responses

obtained in writing or in person

What did they conclude? ⇒ A reception area needed to be established for one of the PALS

services ⇒ Staff awareness of PALS needed to be raised and training for staff

provided ⇒ Links with new partnership organisations were needed ⇒ PALS staff had proved open, supportive and approachable to

patients and Trust staff ⇒ A formal system needs to be developed to record the outcomes of

learning from PALS ⇒ The Shadow Patient Forum should consider all the

recommendations made in the Evaluation Report

Contact: Heather Kelly, Clinical Quality Co-ordinator Email: [email protected]

80

Hull & East Yorkshire Hospitals NHS Trust

PALS EVALUATION: QUESTIONNAIRE TO PATIENTS

The PALS (Patient Advice and Liaison Service) was introduced in April 2001, to provide on the spot advice and support to patients, carers, family and friends when needed. The aim of this questionnaire is to find out how they found the PALS service. All responses will be confidential. Please tick the relevant boxes: ID NO…………… 1. How did you find out about PALS?

Receptionists Complaints Department Other Staff

Leaflet Poster Another Organisation Other (please specify)……………………

2. Did you find it easy to contact the PALS Officer?

Yes No 3. Were you made aware of the option of going through PALS or the Complaints Department

Yes No

4. Did PALS give you the help you needed?

Yes, the issue was sorted out PALS did as much as possible but the issue was not sorted out No, I felt that PALS could have done more

4.a If your problem was not sorted out, did PALS give you advice where to go next? Yes No

4.b. Comments: …………………………………………………………………………………

5. Did you feel that the length of time taken to sort out your problem was acceptable?

Yes No

81

6. Were you kept informed of progress by PALS? Yes No If no, comments please………………………………………………………………………………… ………………………………………………………………………………………………………………

7. How did you rate the service from PALS?

Strongly Agree

Agree Disagree Strongly Disagree

PALS staff were friendly PALS staff were polite PALS staff were professional PALS staff listened to me PALS staff gave me up to date information

8. If contact with PALS was mainly over the telephone how did you feel about this?

Strongly

Agree Agree Disagree Strongly

DisagreeI was happy with telephone contact I would prefer a face to face meeting I would prefer a home visit

9. What did you most like about PALS?

Its accessibility How your enquiry/issue was handled PALS staff attitude towards you How quickly your concern was dealt with If other, please state…………………………………………………………………………

10. What did you least like about PALS? Its accessibility How your enquiry/issue was handled PALS staff attitude towards you How quickly your concern was dealt with

82

11. What hours would you like this service to be available?

24 hour telephone service Reception cover Mon – Fri 9am to 5pm

Reception cover evenings and weekends

Other, please state ………………………………………………………………………..

12. Is there anything about PALS that you would like to see changed or improved? …………………………………………………………………………………………………. ………………………………………………………………………………………………….

13 Would you use the PALS service again for a similar problem?

Yes No

14. Would you recommend someone else to use the PALS service?

Yes No

15 Were you asked if you would like to be involved in the work of the Hospital? Yes No

15a, If yes, have you actually been involved in any of the following:

Participate in a user group Survey Steering Group Advisory group User panel Other If other, please state ………………………………………………………………….

16. Any other comments: …………………………………………………………………………………………………. …………………………………………………………………………………………………. ………………………………………………………………………………………………… …………………………………………………………………………………………………

THANK YOU FOR TAKING THE TIME TO FILL IN THIS QUESTIONNAIRE

83

ID NO…… PALS Staff Awareness Questionnaire

PALS (Patient Advice and Liaison Service) was introduced in this Trust as a Pathfinder in April 2001. All NHS Trusts should have PALS by April 2002. In order to evaluate the service, we would be grateful if you could take time to complete this questionnaire. In addition, we are conducting a Patient Satisfaction Survey, in order to obtain their views. All responses will be confidential and feedback from this survey will be available on the Trust Intranet site and in the Newsletter. On completion, please return in the envelope provided to: Name, (address) by (date) .

JOB TITLE: Consultant Nurse/Service Manager Admin & Clerical/Secretaries Nurse Other If other, please state …………………………..

1a. Are you aware of PALS? YES NO 1b. If yes, how did you find out about the service? Newsletter Meetings Presentations Leaflet Posters

Other ……………………

If no, please contact 3065 CHH for further information 2. Do you feel you understand the role of PALS? YES NO 3a. Have you contacted PALS? YES NO

If no, please go to question 4a

3b. If yes, please state reason:

To refer patient Advice/information User Group Training

Other ……………………..

3c. What made you decide to use PALS?

…………………………………………………………………………………………………..

………………………………………………………………………………………………….

PLEASE TURN OVER

84

3d. Did you have any problems accessing PALS? YES NO 3e. If yes, please state…………………………………………………………………… 4a. Have you been involved with a PALS issue? YES NO

If no, please go to question 5 4b. If yes, what was your overall experience? POSITIVE NEGATIVE 4c. How many times have you been involved with a PALS issue? ………………..

5. Are you clear about the differences between PALS and complaints?. YES NO 6. What do you feel that PALS do well? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 7a. Have you any ideas/suggestions on how to improve the service? YES NO

7b. If yes, please state ………………………………………………………………… ………………………..………………………………………………………………………… 8. Do you have a supply of PALS leaflets on your ward/department? YES NO 9. Would you be interested in attending a PALS training session?

YES NO 10. Any other comments ………………………………………………………………. ………………………………………………………………………………………………….

THANK YOU FOR TAKING THE TIME TO COMPLETE THIS QUESTIONNAIRE

pals core national standards and evaluation...

Documents