insert organisational logo here on master slide techniques

Insert organisational logo here on master slide

TECHNIQUES


REMEMBER

All projects should have – specific aims, – declared outcomes– appropriate methods– sufficient resourcessufficient resources (money, people

and time)– should be evaluated at the end point.


What techniques do you use?

• Resources and time will often dictate what you can do.

• Quantitative = How many people agree/disagree with the changes

• Qualitative = Why they agree/disagree


TECHNIQUESQualitative and Quantitative

• Not either/or but mix and match• There is a need for Qualitative

and Quantitative to give a complete picture

(This is often a luxury)• Whatever technique used beware

of BIAS. Trade off between bias, convenience and pragmatism


Many problems will be avoided if a PILOT

is done first

• A pilot is a small scale replica of the main survey

• All parts of the process must be piloted• Be prepared to make the changes• Re-pilot if necessary


•Be brave, abandon rather than waste everyone's time


What techniques could we use? (1)

• Open Public Consultation– Meetings– Websites– The Media– Open Days, Road shows

• Citizens’ Panel• Public and/or Patient Forum

– Standing Panels/Health Panels

• Patient/Lay representation on specific groups– Reference group– Liaison Group– Management Group– User Group

• Focus Groups


What techniques could we use? (2)

• Surveys– Interview– Questionnaire

• Patient stories/ Patient diaries• Process mapping• Observations of care• Compliments/ Complaints• Staff feedback• Audit• Community Development Initiatives• Participatory/Rapid Appraisal


Who will

• collate the responses?• be responsible for quality control, coding and data entry?• undertake the analysis and write the report?• develop any recommendations and action plan?• ensure the changes happen?• disseminate the results/feedback?


WHO DO WE ASK?

• Everyone– Patients/Users– “Public”

• Community Health Council• Voluntary Sector


WHO DO WE ASK?

• A sample

• Representatives

• All patients

• “Nice” patients

• “Stroppy” patients

• Colleagues

• Friends – ours– patients’

• Relatives– ours– patients’


HOW?

• Identify whole population of interest

• What characteristics/variables might affect findings (age, gender etc)?

• Try to identify a true random sample

- convenience samples usually have bias (pub closing time in A&E etc)


DO NOT FORGET

ETHICAL APPROVAL

DATA PROTECTION

INFORMED CONSENT


PATIENT SATISFACTION SURVEYS HAVE A BAD NAME!

Why?

• No clear purpose behind many in the past• Superficial• No changes made

BUT they can be very useful and informative


Benefits of Questionnaire Surveys

• Cheap• Can be done by one person but can separate

design, data handling etc• Repeatable• Cover wide geographical area• Larger sample• Observer bias minimal (take care with question

design)


Drawbacks with Questionnaire Surveys

• Low response rate - reminders may be required

• Time lag between questionnaires out and returns

• Sample bias, literate, language etc

• No clarification possible, and the qualitative bit often missing

• Difficult to inform respondents what the outcome was, need to “tell the world”

• Data collection and interpretation may be difficult


Benefits of interviews

• Better response rates• In depth – qualitative• Flexible, can follow up leads• Questions can be clarified• Fewer unanswered questions• Interview questions can be refined during the process if

unexpected findings arise or focus on specific problems if necessary

• Tape with permission• ~1 hour


Drawbacks of interviews

• Social and ethnic characteristics of interviewer may affect responses

• Expensive, transcribing time ~8:1• Coding difficult• Right answers• May need interpreters• Need to make appointments


Benefits of group

• Short time scale• Can work with only one skilled person • Can repeat with cohort, before, during and after service

changes• Can include those with low literacy skills• Stimulates debate• Good at identifying cultural values• Gets those with nothing to say involved• Good back up qualitative for quantitative


Drawbacks of groups

• Not representative, volunteers are self-selecting and may want to “hijack” the group

• Can change ideas during discussion when more information available

• Difficult if transport an issue!• Transcription and coding can take a long

time


Benefits of Public meetings

• Everyone and anyone invited to attend

• All attendees invited to contribute• Potential for large numbers of

people to become involved


Drawbacks for Public Meetings

• Meeting “hijack”• Many people unable/unwilling to speak out in

public• Many staff often required to man event• Difficult to give sufficient background

information in the time available • Poorly attended


Whatever technique used, the questions posed must be

• relevant• unambiguous• give some simple options• in appropriate language


Questions must not be

• offensive• biased• leading• full of assumptions


EVALUATION

• Comprehensive report available – Results presented in understandable format

• All stages described, aim, method etc• Method

– Justified– Repeatable

• Outcome measures specified• Discussion of biases etc• Context of other work given• Recommendations follow from results


WHAT COULD BE IMPROVED ABOUT THE METHODS?

• What facilitated/inhibited involvement?• What influenced implementation of

recommendations?• What benefits/drawbacks were there for those

who became involved?• Who else could/should have been involved to

increase information or changes made?


INFORMATION USED EFFECTIVELY?

• What changes were made?• Were the respondents consulted about

the changes?• Were there conflicts of interest?• Dialogue continued between

professionals and public• Could the results be generalised?

insert organisational logo here on master slide techniques

Documents

bias pub

firsta pilot

specific aims

data entry

people agreedisagree

ae etcpublic

small scale replica

waste everyones timepublic