evaluation update pssru personal social services research unit at the university of manchester
TRANSCRIPT
Evaluation Update
PSSRUPersonal Social Services Research Unitat the University of Manchester
PSSRUPersonal Social Services Research Unitat the University of Manchester
2
Introduction
1. The task set
2. Our approach
3. Progress
4. Next steps
PSSRUPersonal Social Services Research Unitat the University of Manchester
3
The task set by DH
Map the different approaches to developing CAF Provide evidence about the quality of outcomes for
service usersProvide evidence about workforce experienceProvide evidence on the cost, effectiveness and cost-
effectiveness Provide evidence about the usability of the CAF IT
systems for practitioners and service users Disseminate and share widely the learning
Identify impact of Improving information communication
PSSRUPersonal Social Services Research Unitat the University of Manchester
5
Key issues
Issues Providing evidence from each
site that relates to the study objectives set in the brief while being sensitive to the different approaches being adopted in each area
Developing a Routine Data Set that is relevant to each site and which can be enhanced by additional local material that is specific to particular sites
Recognition of the different target groups under consideration in each site
Response Overarching evaluation
framework – covering both a) process of development and b) service delivery and outcomes Multiple case study design,
not a single approach … informed and agreed by
discussion with each site Ultimately signed off by the
steering group and National Research Ethics Committee
PSSRUPersonal Social Services Research Unitat the University of Manchester
6
Two broad strands
process and implementation – the process of setting-up and executing the programme and the subsequent cultural change that occurs as a result Led by SQW Intended to be fieldwork at regular intervals
service delivery and outcomes – the resultant service user, carer, and workforce engagement and subsequent outcomes that are experienced as a result of participating in the programme Led by PSSRU, with support from GfK Tailored approaches to reflect different user groups and timings
PSSRUPersonal Social Services Research Unitat the University of Manchester
7
Key tasks undertaken to date
Introductory visits to sites Refinement of research framework and design of
research toolsEthical approval and research governance arrangements
PSSRUPersonal Social Services Research Unitat the University of Manchester
8
From site visits
The wide range of activities being pursued Some areas much clearer in their approaches than others
Conceptualising what they are doing that we can evaluate Focussing on where IT is used (not paper) and users/ carers benefit
… and on the scale of activity But generally it appears lower than we anticipated Especially in the time period
The challenge in defining what a ‘go live’ date actually was Often incremental development (paper, local IT, national IT) New systems rolled out to staff before users/carers
Go live dates subject to change
PSSRUPersonal Social Services Research Unitat the University of Manchester
9
Research tools
Process and implementation Individual site case study
report outline Topic guide for project leader
consultations Topic guide for staff focus
groups Topic guide for user
representative consultation Topic guide for consultation
with IT professionals Implementation costs – staff
and money
Service delivery and outcomes Service users routine data set Staff well being and time use Service user assessment
satisfaction survey Service user and carer
outcomes Carer outcomes Engagement with new
technology Assessment document
analysis Assessment pathway
Several Hoops to pass through
PSSRUPersonal Social Services Research Unitat the University of Manchester
11
Ethics and research governance
National application submitted in SeptemberAttended committee in OctoberFormal clearance November
Now working through local arrangements Require both Local Authority and NHS clearance As yet not fully formally agreed in any site We would really welcome your help in speeding things along
PSSRUPersonal Social Services Research Unitat the University of Manchester
12
Local discussion – working towards
Identify settings where CAF influences assessment and care pathway for study
Identify agreed local NHS contact/collaborator for research (required for Res Gov)
Research Governance in place LA and NHS (within agreed National REC parameters)
Agreed routine data set collection processLocal plan for data collection from users/carers
experiencing CAF processes and similar people not doing so
Local agreement for access to a range of staff for interviews in NHS and LA
PSSRUPersonal Social Services Research Unitat the University of Manchester
13
So that we can…
Begin collecting user/carer data as soon as possible Especially as we know in some cases users are beginning to
experience the new approaches
Map out the delivery experience and lessonsDescribe and evidence the legacy of the CAF
programmeEnsure the lessons can be shared widely with interested
parties
One CAF experience
PSSRUPersonal Social Services Research Unitat the University of Manchester
15
Contact
Graham ThomAssociate DirectorSQW Consultingt. 0131 225 4007e. [email protected]. www.sqw.co.uk
Prof David ChallisDirectorPSSRUUniversity of Manchestert. 0161 275 5250e. [email protected]
Claire IvinsDivisional DirectorGfK NOP Social Researcht. 020 7890 9783 e. [email protected]