no place like home rapid city evaluation training
TRANSCRIPT
No Place Like HomeRapid City Evaluation Training
General Staff Survey
• Please open your browser to: https://www.surveymonkey.com/s/RAPIDSTAFF
• Read the informational page, and if you agree to take the survey, please click “Next” to begin
• All of your responses are CONFIDENTIAL• It should take approximately 15 minutes to
complete
NPLH Summary
• Evaluation of three child welfare agencies’ use of family meetings – Process
• What does practice look like?• Tools: focus groups, observations, surveys
– Outcomes• Child welfare placements (number and type), re-
reports, recurrence of maltreatment, etc.• Family functioning and protective factors• Satisfaction and service provision• Tools: administrative data, surveys
Your role…
• Is critical!• To provide data and feedback about your
experience working in child welfare and with family meetings– As part of a federally-funded study that will be
disseminated nationally• To invite families to participate, so that they
can provide data and feedback about their experience
• To participate in the feedback loop – ongoing communication
Who What is the value?
…the field •Research is mixed and the evidence-base of FGDM is moderate—more research is needed•Additional research can support the national spread of family meetings or suggest the need for alternative practices
…families •Families receive interventions with known effectiveness•Opportunity for families to provide input about their experiences with CPS
What’s in it for…?
Who What is the value?
…CPS Staff and Agencies (You!)
•Accountability. •Opportunity to get input from staff who know the most about the families (You!)•Knowledge of effectiveness of family meetings can increase investment in the practice•Visibility and respect for advancing the field and systematically examining internal practices•Increased knowledge about the use and potential of evaluation•Results can impact system and practice change
What’s in it for…?
Human Subjects
• Overview–History of human subjects research–Historical laws and documents
protecting human subjects–Your role in protecting the rights of
participants*
Human Subjects History
Before there were human subjects protections:
• Nuremberg (1930’s – 1940’s)– Concentration camp prisoners, twins
• Willowbrook Hepatitis Study (1956)– Mentally retarded, institutionalized children
• Brooklyn Jewish Chronic Disease Cancer Study (1963)– Poor, eldery
• San Antonio Contraception Study (1971)– Mostly poor, women
• Tuskegee Syphilis Study (1932-1971)– Poor, black males
Human Subjects History
Attempts to protect human subjects:• The Nuremberg Code (1947)
– Informed consent. Benefits outweigh risks. Voluntary participation.
• Declaration of Helsinki (1964…2000)– “Concern for the interests of the subject must always prevail
over the interests of science & society”
• National Research Act (1974)– Tuskegee Institutional Review Board (IRB) approval required if
getting U.S. Dept. of Health and Human Services funds
• The Belmont Report (1979)– “Justice, beneficence, respect for persons”
Our Role
Our Role in Protecting Participants Rights
• Give participants all the information that they may need to make an informed decision.
• Ensure that participants understand the information• Give participants an opportunity to consider
involvement in the project on an initial and ongoing basis
News You Can Use
• What’s important to know:– Risks to you personally are minimal– There are potential benefits to children, families,
and to staff in the agency – Your participation is voluntary
• Privacy and Confidentiality– All survey information is confidential and used
only for the study– Survey data is maintained by Kempe Center
NPLH evaluation team– Individually identifiable information will NOT be
shared outside of the NPLH evaluation team
Evaluation Terms
• Evidence-Based Practice• Confidentiality vs. Anonymity• Bias• Model Fidelity• Validity• Qualitative vs. Quantitative methods
–Depth vs. Breadth
Evaluation Design - SD
General Staff Survey
Families meet eligibility criteria and are notified about the study; Caregiver Survey
Meeting participants notified about the study; Fidelity Survey
Follow Up Survey
Case-Specific Questionnaire
Right now
Original study cases:IFA specialist makes FGC referral at transfer to Ongoing (in-home services)
FGC
1-2 months post-FGC
Ongoing case closure/transfer to out-of-home services
Concurrent Planning Cases: Ongoing specialist makes FGC referral for cases where reunification within 6 months is likely (out-of-home services).
Ongoing case closure or TPR
NPLH Surveys
1. General Staff Survey
2. Caregiver Survey
3. Meeting Fidelity Surveya) Participant
b) Coordinator
4. Case-Specific Questionnaire
5. Follow-Up Surveya) Caregiver – Intervention
b) Caregiver – Control
c) Meeting Participant
Surveys that are distributed TO you
• General Staff Survey–All staff who have a role in the
evaluation (family service specialists, coordinators, supervisors)
• Coordinator Meeting Fidelity Survey–Coordinators only
• Case-Specific Questionnaire–Ongoing specialists only
General Staff Survey
• Purpose:–To understand more about:
• Who utilizes family meetings in CPS• How much experience staff have with
family meetings• How staff perceive family meetings• The context in which meetings are
happening.–We can use this information to inform
our process and outcome evaluations
General Staff Survey
• Who? You!– IFA and Ongoing specialists providing in-
home services– Family meeting coordinators– Supervisors of both
• When? Now
• How? Via Survey Monkey
Coordinator Fidelity Survey
• Purpose:–To understand more about:
• How, why, and when FGCs are used• Who is invited to and who attends FGCs• Other characteristics of family FGCs
–Fidelity–Logistics
–Meeting fidelity is part of our process evaluation
Coordinator Fidelity Survey
• Who? Meeting coordinator
• When? After the FGC – Once back at your desk/computer (same day
or day after)
• How? Via Survey Monkey
Case-Specific Questionnaire
• Purpose:–To understand more about:
• What services in-home families receive• Are services provided impacted by family
meetings?–Services provided is one of our outcome
evaluation variables
Case-Specific Questionnaire
• Who? Ongoing Family Service Specialists• When? At case closure OR transfer from
in-home services to out-of-home services (foster care, kinship care, etc.) OR TPR– Transfer does NOT mean voluntary
placements– It does mean a formal or legal change in
custody• How? Via Survey Monkey
– We will send you a ‘tickler’ or reminder
Surveys that are distributed BY you
• Caregiver Survey– By IFA or Ongoing family service specialists– At face-to-face meetings with family– At point of referral for an FGC
• Participant Fidelity Survey– By coordinators– At NPLH FGCs
What you ARE responsible for
Distribute:1. Information Sheet
• Explains project to participants.• Consent is not required but participation is
voluntary.• What is voluntary? Completion of surveys, NOT
participation in child welfare services.
2. Paper and pencil surveys with:• Caregiver Survey – self-addressed and stamped
envelope with a flyer with a phone number to call for assistance
• Meeting Participant – large sealable envelope for all surveys in the center of the room/table
What you are NOT responsible for
• Explaining the content of surveys to families– People with questions can call the number provided
on the Information Sheet (have extra copies available for them to keep)
• Helping families/meeting participants fill out surveys– Other household members or family supports, etc.
can help. • In order to maintain confidentiality, it is
important that you merely handout surveys and provide no more information than what is included on the information sheets.
Caregiver Survey
• Purpose:–To understand more about:
• The characteristics of families receiving in-home child welfare services (e.g. demographics)
• The protective capacities of parents • The parent-child relationship and child
behavior–This information can be linked to
outcomes
Caregiver Survey
• Who? Parent or legal guardian• When?
– Face-to-face meeting– At time of referral for an FGC
• How?– Paper and pencil– Self-addressed and stamped envelope– Distributed by IFA OR Ongoing specialist– Parents can call the number on the information
sheet to take the survey via phone
Participant Fidelity Survey
• Purpose:– To understand more about:
• How, why, and when family meetings are used• Who is invited to and who attends family meetings• Other characteristics of family meetings
–Fidelity–Logistics
– Meeting fidelity is part of our process evaluation
– We want to hear from families and other participants as well as staff.
Participant Fidelity Survey
• Who? Any meeting participant who agrees to fill it out
• When? AT the FGC– Section 1: Before the meeting starts– Section 2: After the meeting ends
• How? Paper and Pencil– Meeting participants will fill out paper-pencil
surveys and place them in a large envelope– The meeting coordinator will collect the
envelope and bring to a safe spot to be mailed to the evaluation team
Surveys administered by evaluation team
• How? Paper and pencil; via mail• 2 follow-up survey versions with the
following components:– Meeting Follow-Up
• Who? All meeting participants who filled out the initial Meeting Participant Fidelity Survey
– Caregiver Follow-Up• Who? All parents
– Family Satisfaction• Who? All parents AND meeting participants
Coordinator Meeting Log
• Coordinators:– You will be asked to fill out information an
additional “Meeting Log” spreadsheet • Ex. Date of referral, meeting type, date of meeting,
meeting cancellation reason, etc.
– This will be posted on a document sharing SharePoint website to which you will have access
NPLH Survey Summary
• IFA and Ongoing Family Service Specialists:– Complete:
• General Staff Survey (1 time)• Case Specific-Questionnaire (for every study case)
– Ongoing only
– Distribute:• Caregiver Survey (to every study case)
• Coordinators:– Complete:
• General Staff Survey (1 time)• Coordinator Fidelity Survey (for every study FGC)• Meeting Log (for every study FGC)
– Distribute and Collect:• Meeting Participant Fidelity Survey (for every study FGC)
Where can I find survey materials?
Michelle Lisa Fleming’s secretary
– You can find extra survey materials here• Caregiver surveys with self-addressed, stamped
envelopes• Extra information sheets• Meeting Participant fidelity surveys and envelopes
– Coordinators can drop off survey packets here after meetings for bulk mailing at a later date
– When you are running low on materials, we will deliver more
When do we start?
Original (in-home) study population: 10/8/2012
New (concurrent planning) population: 4/2013
THANK YOU!
If, throughout the life of this
project, you have any questions
please contact:
Heather Allan, NPLH Project Coordinator
Email: [email protected]
Phone: 720.336.8283