from training to practice transformation
TRANSCRIPT
FROM TRAINING TO PRACTICE TRANSFORMATION
Implementing a public health parenting program
Sally Gaven, Evaluation Project ManagerJanet Schorer, NSW Department of Family & Community Services
Nexus Evaluation Team: Greg Masters, Sally Gaven, Dr Louise Askew, Ashley Pennington
Professor Sven Silburn of Menzies School of Health Research in Darwin,Professor Sven Silburn of Menzies School of Health Research in Darwin,
A/Prof Stephen Jan of The George Institute for International Health, University of Sydney
Agenda 1. Since when did parenting
become public health?
2. Evaluating the population impact of a parenting programprogram
3. What changed? • In the system• In the system• For families
4. The 21st century challengey g
5. 5 steps to practice transformation
6. Q&A
A public issue
Childhood conduct problems
Increased risk for:• Crimeconduct problems• Substance abuse• Mental health
bl d i idproblems and suicide• Domestic and family
violenceviolence• Poor physical health• Early parenthood
No other common childhood condition with such far-hi
y p
reaching consequences.Christchurch Health & Development Study (Fergusson 2010)
Triple P
Parenting intervention aimed at reducing the risk factors for poor developmental outcomes in children.
By: Professor Matthew Sanders and colleagues at University of Queensland’s Parenting and Family Support Centre -Bandura’s cognitive social learning theory 1977, 1986
Improves: and prevents behavioural, emotional and developmental problems in children – by improving knowledge and skills of parents
Evidence: By far the most successful intervention for childhood conduct problems.
The Triple P system of flexibleThe Triple P system of flexible intervention and reach
Breadth of Reach
Flexible Delivery FormatsIncreasing Intensity
Flexible Delivery Formats
Individual Self-directed Group
Low Parenting Information Campaigns
LEVEL 1Brief Parenting Advice
LEVEL 2Narrow Focus Parent Skills Training
LEVEL 3Broad Focus Parent Skills Training
LEVEL 4Behavioural Family Intervention
Source: Sanders, M.R. & Prinz, R.J. (2005) The Triple P System. The Register Report Spring: 42-46.
High LEVEL 5
Population-level, public health approach - a coordinated system of training and accreditation for
titipractitioners:
• Over 62,000 practitioners trained
• From Australia, New Zealand, Singapore, Hong Kong, Canada, United States, England, Scotland, Belgium, The Netherlands, Curacao, Republic of Ireland, Japan, Germany, Switzerland, Sweden and Iran
• Across various fields of health, education, childcare, general practice and social welfare.
Triple P in NSW
Families NSW: range of prevention and early intervention initiatives to improve the health and well-being of families
ith hild f bi th t i htwith young children from birth to eight years.
jointly delivered by:NSW Health
Department of Family & Community Services
Department of Education and Communities
In partnership with families communityIn partnership with families, community organisations and local government.
Long term aims: children’s developmentLong term aims: children s development
Levels 2 and 4: seminars and groups
Reduce risk factors:• The prevalence of early onset behavioural and emotional
problems in childrenproblems in children • Coercive, harmful & ineffective parenting • Parents’ emotional distress and conflict
Increase protective factors:• Parental confidence and efficacy• Positive parenting practicesp g p• Participation in evidence-based parenting programs
And to build the capacity of communities to support parents:And to build the capacity of communities to support parents: • Capacity and confidence of service providers
• Interagency collaboration and referral pathways
Implementation of Triple P via Families NSW
To offer free parenting training to all NSW parents with children 3-8 years.
Led by Community Services - $5m in Government funding over four years to:to:
• Governance via Triple P Working Groups
• Train 1,100 practitioners from government and non-governmentTrain 1,100 practitioners from government and non government agencies 2008-2010
• Support delivery by each accredited practitioner (2 x Seminar Series & 2 G o ps pe ea ongoing)& 2 x Groups per year, ongoing)
• Collect data on delivery for program management and evaluation
• Support networks of practitioners• Support networks of practitioners.
Governance Training Delivery Data NetworksGovernance Training Delivery atacollection Networks
The tri-fold evaluationquantity and quality of outputs• Program data – practitioners, courses, attendees, demographics• Focus groups – partners & over 60 practitioners
P titi
PROCESS:
• Practitioner survey
quantity and quality of changes in child behaviour
• Non-randomised controlled trial• Non randomised controlled trial• Pre and post outcome data • Family survey
quantity and quality of changes in parenting practices
OUTCOME
quantity and quality of changes in parenting practices
• Family survey • Attendee Satisfaction Survey
costs and benefits of creating these changes
• Costing analyses – head offices, regional offices, providersECONOMIC
longer-term population impacts
• Literature review, emerging implementation science
ECONOMIC
Outcome evaluation methodsQuestion MethodQuestion Method
OverallWhat are the short and longer term effects of Triple P on parents and children?
Level 2: quasi-experiment
Level 4: pre and post scores analysis
Family interviews
Attendee satisfaction data analysisWhat are the outcomes for nominated target groups? Level 4: pre and post scores analysis
Family interviewsParentsWas there an increase in use of and confidence in positive parenting behaviours?
Family interviewspositive parenting behaviours?Was there a decrease in coercive, harmful or ineffective parenting behaviours?
Family interviews
Was there a decrease in emotional distress over parenting?
Family interviewsparenting?Was there a decrease in parenting conflict? Family interviewsDid the program meet participants’ needs? Attendee satisfaction data analysis
Family interviewsyWere there unintended consequences? Family interviewsPopulationWhat were the population-level impacts? Literature review
Quasi experiment study design
A1 A2Triple P group
( 104)A3
(n=104)
Comparison group (n=78)
B1 B2 B3
Prescore
Postscore
6 month follow-up
Section 5ChallengesChallenges
Section 4 Section 6Outcomes Impacts
Section 7Section 3Output
Section 7Strengths for Development
Section 2Overview
Section 8Future Lessons
FNSW Triple P Evaluation: Final Report
What was done? Outputs
Program and client demographic and satisfaction data to examine the quantity and quality of:
• The practitioner training
Th T i l P• The Triple P courses delivered to parents by practitionerspractitioners
• The support for practitioners available through Familiesavailable through Families NSW.
Practitioner Training
How much?
• In just over two years, over 1,000 practitioners were trained to deliver Triple P.
• Two thirds are from 250 non-government organisations.g g• Of the Government practitioners, only 19% are from the Department of
Education and Communities.• Most practitioners are child and family workers, caseworkers or managers.
100 d li T i l i l th th E li h• 100 can deliver Triple in languages other than English.
How well?
• 86% of trained practitioners achieved accreditation.N l ll titi f lt fid t b t d li i T i l P ft
How well?
• Nearly all practitioners felt confident about delivering Triple P after training.
Course delivery to parents
1 257 T i l P d li d b th d f 2010 92 i th
How much?
• 1,257 Triple P courses were delivered by the end of 2010 - 92 in other languages.
• Only 60% of trained practitioners had started delivering courses to families.
• Only a third are delivering the expected number of courses.• An estimated 12,500 attendees came to Triple P by end 2010.• These attendees are generally more disadvantaged than the general
pop lationpopulation.• Their children experience more emotional and behavioural difficulties.
H ll?
• High client satisfaction• Program fidelity tools not consistently used – although changes mainly
How well?
• Program fidelity tools not consistently used although changes mainly suit client needs
• Broad engagement strategies not widely used.
Practitioner Support
Implementation science shows that high-fidelity, effective and efficient practitioner delivery needs implementation d i th t fdrivers that focus on:
• Targeted practitioner selection
• Training
• Ongoing coaching consultation and supportOngoing coaching, consultation and support
• Data systems to support decision-making
( ll b d• System interventions (collaboration and peer support –international experience).
Focus of the 2010 issues paperCommunity Services initiated several specific actions to support practitioners:
• Triple P Working Groups – patchy
• Peer Support Groups – mentoring, co-facilitation
• Annual Practitioner Development Day - 300 practitioners from NSWacross NSW
• Scoring application to improve attendee demographic and outcome data entry by practitionersy y p
• Practitioner website promoting practitioner networking and collaboration
• Promotional resources - to broaden, streamline and standardise engagement strategies
A i t f di t t T i l P titi ( i l• Assistance funding to support Triple P practitioners (regional practitioner forums, AV equipment, childcare, refreshments and venue hire).
What changed? Outcomes
Level 2 – quasi experiment
• Children of sampled families attending Triple P showed C d e o sa p ed a es a e d g p e s o edsignificant improvements in behaviour (in contrast to children in the non-treatment comparison group).
• Quasi-experiment: only the Triple P group showed a Q p y p g pmarked improvement six months after doing a Seminar Series compared with the no-treatment
f b h b d lcomparison group – for both boys and girls.
• 10% net reduction in children in the clinical range.
Q iQuasi:SDQ Total Problems Score - Means and 95% Confidence Intervals
Sc
ore
s
6 month follow-up
6 month follow-up
Pro
ble
ms S
SD
QTo
tal P
S
Triple P Comparison
NOTE: the blue, green and red lines refer to the 95% confidence interval and the dot in the middle shows the mean
Triple P group clinical status:Triple P group clinical status: SDQ total problems scorepre score6 month follow-up
88.2100
78.575
% SDQ scores
21.525
50% SDQ scores
clinical/ normal
11.8
0
25
Pre-intervention 6 month follow-up
Clinical range Normal range
Level 4: Pre and Post scores
Level 4 attendees reported significant improvements in:
• Parenting behaviours (Parenting Scale)
• Child behaviours and emotional difficulties (SDQ)
Over 11% net reduction in the proportion of children in the SDQ clinical range.
Level 4: Clinical status on SDQ total problems score pre and post-intervention
100
p ob e s sco e p e a d post te e t o
64.775.2
75
% SDQ scores
35.324.8
25
50% SDQ scores clinical/ non-
clinical
0
25
P P tPre-program Post-program
Clinical range Non-clinical range
Family surveys
Most respondents felt, following Triple P:
• Their child’s behaviour had got better
• They had changed their parenting practices
• More confident in their parenting.
Challenges
However, the evaluation articulates two key challenges in d li i l ti b ddelivering a population-based parenting program:
• Translating training into delivery of the program to familiesfamilies
• Achieving reach into the population by engaging apopulation by engaging a sufficiently broad range of families.families.
The evaluation has demonstrated thatThe evaluation has demonstrated that the implementation has, so far:• Successfully trained a multi-disciplinary workforce
• Enabled high quality delivery
• Achieved positive results for the clients it has reached
However:• Too few of those trained are delivering coursesToo few of those trained are delivering courses
• Only a fraction of the planned courses are being delivered
• Fewer families than expected are attending• Families attending are not representative of the• Families attending are not representative of the
general NSW population.
1,180
TARGET
on trackpractitioners trained:
ACTUAL
ll i 2 i & 2 b /3 hi iFull-time: 2 seminars & 2 groups Part-time: 1 seminar & 1 group
expected to deliver:
about 1/3 achieving expected delivery
to achieve a total of:
1,674 Seminars per year1,674 Groups per year
In 2010:301 Groups
287 Seminars
to reach: 300,000 familiesBy end 2010:
12,580 familiesto reach: 300,000 families 12,580 families14% of expected reach
to result in:improved family and population outcomes
improved family outcomes
ImpactsCosts
• Direct investment $5M - leveraged $8M value
• Cost per child $641 – high because early in implementation?
$ %
Estimated total cost of implementation of Triple P in NSW
Head office 4,744,820 59.2
Partner agency 86,000 1.1
Regional co-ordination 663,408 8.3
Delivery 2,520,625 31.4
Total overall cost 8,014,853 100.0
ImpactsPractice impacts
• Doing a better job
• 90% would recommend Triple P
• 90% said it enhanced client
• collaboration benefits
• But – at the major cost of time pressure, for time-poor practitioners!ut at t e ajo cost o t e p essu e, o t e poo p act t o e s
Population impacts
• Evidence of longer-term social benefits and reduced costs from• Evidence of longer term social benefits and reduced costs from Triple P
• Extrapolating the outcome data, Triple P in NSW has already moved p g , p y1,150 children from the clinical to the non-clinical range.
Wh h Th t hTraining a pool of
practitioners
Who have delivered high quality, well-i d
That have achieved good
results for families d hildreceived programs and children
H th i l t ti i t t hiHowever, the implementation is yet to achieve:
• Expected delivery rates
• A viable population reach.
Practice transformationThe transition from science to service
The science of implementation reveals the magic ingredients that power d b d d l levidence-based service delivery to population impact.
Program: Population impact:
Evidence based Implementation drivers that transform practice:
High fidelity Targeted participation
Effective Ongoing support
Efficient Data-driven decisions
Collaboration(Fixen & Blasé 2009)
Fergusson et al 2011
Apply the prevention science paradigm to program implementation:
• Select evidence-based programs
• Pilot - program acceptability and fidelityPilot program acceptability and fidelity
• Randomised trial - program efficacy at the new site
T k t l i l it• Take to scale progressively - monitor program effectiveness.
The 21st Century Challenge:
Strong body of evidence for effective treatment of conduct disorders BUT implementation barriers:
• Policy makers Researchers
• Resistance to change• Resistance to change
• RCT phobia
Triple P in NSW has cleared most of these barriers:
• Trained over 1000 practitioners in an evidence-based a ed o e 000 p a o e s a e de e basedprogram
• Tested the early implementation: quasi-experimentTested the early implementation: quasi experiment
• Policy-research engagement.
5 steps to practice transformation
The hurdles ahead: establishing the infrastructure to support delivery and achieve population reach.
Strengths to be developed:
• Universal entry pointsUniversal entry points
• Broad and strategic program promotion
I t t d d t ll ti ( t t d li t• Integrated data collection (program output and client outcome data)
l d l• Active central delivery management
• Concerted and consistent practitioner support at all levels.
Strength 1: Universal engagement
Ron Prinz (US Triple P Population Trials) identifies four key features of the population approach to implementing Triple P d f di blP – and four corresponding enablers:
Features of the population Universal engagement eatu es o t e popu at oapproach
U e sa e gage e tstrategies
1. Aiming for population impact Mass media campaigns
2. Using existing multi-disciplinary workforce Interagency collaboration
3. Broad availability and accessibility Universal entry points
4. Sources of efficiency Inclusivenessy
Population-level impact enabler = mass media campaign
Recommendation: mass media campaign
N t j t i i
Key messages
• All parents need information d t t b th b t th• Not just awareness-raising -
depict practical parenting solutions
and support to be the best they can for their children
• Going to parenting courses can• Address stigma (‘listen up bad
parents’)
• Going to parenting courses can make a huge difference to children’s development and i f il lif• Create positive norms
• Drive program engagement.
improve family life
• Parenting information and support is helpful and easy to
Strengths
support is helpful and easy to access at the FNSW website.
• Triple P has a mass promotional component• Families NSW website.
g
M lti di i li kf bl i tMulti-disciplinary workforce enabler = interagency collaboration
Recommendation: active interagency working groups – with funds
Without genuine, proactive engagement at the regional Working Group level, driven by the FNSW senior officers group, the d li d h f T i l i lik l t idelivery and reach of Triple is unlikely to improve.
Involving Working Groups in the expenditure of the annual f d dassistance funding, to support practitioners and promote
collaborative delivery, would help engage these groups.
Strengths
• Families NSW structures promote interagency collaboration• Community Services is raising Triple P’s profile as a central Families NSW
intervention.
B d il bilit d ibilit bl i l tBroad availability and accessibility enabler = universal entry points
The literature focuses on using universal entry points –specifically:
Recommendations to:• Integrate Triple P with transition
to school programsspecifically:• An education sector that
embraces and funds parenting t i h l tti d
to school programs• Promote Triple P internally with
Department of Education & support in school settings, and
• Healthcare settings that offer parenting support.
Communities• Make direct approaches to
principals and P&Cs to promotepa e t g suppo t
However, less than one fifth are from the Department of Ed ti d C iti
principals and P&Cs to promote Triple P
• Neutral venues - deliver through local government services andEducation and Communities.
Strength
local government services and libraries.
• Good take-up of Triple P by health practitioners, delivering Triple P in community health settings.
g
Sources of efficiency enabler = inclusiveness
Creating a context where the population approach belongs to all of the participating professionals, the parents, the entire communitycommunity.
Recommendations to broaden engagement strategies beyond the welfare sector:welfare sector:
• Service specifications in funding agreements to support broad community engagementy g g
• Promotional activities with a universal reach – beyond service clients.
St th
• Very high level of enthusiasm for Triple P from delivering practitioners.
Strength
Strength 2: Practitioner support andStrength 2: Practitioner support and collaborationOngoing practitioner engagement relies on:
• Completing the full training process, including accreditation p g g p , g
• Practitioner’s confidence following training
• Time between training and delivery• Time between training and delivery
• Organisational support for practitioners Seng, Prinz, & Sanders, 2006; Sanders, 2008.
We identified 4 sources of support in NSW: Recommendations for:
• Community Services (and Families NSW) Ongoing funding
• Service providers Delivery accountability
• Peers Active Peer Support Groups
• Collaboration Practice networks for co-delivery
Strength 3: Data collectionImplementation is optimised when:
D t di d ti
Recommendations:• Ongoing funding dependent
d t ll ti• Data recording and reporting is embedded
• Data covers the input-
on data collection • Track delivery and enforce
accountabilityData covers the inputoutput-outcome spectrum
• Data is used to track delivery
accountability• Annual practitioners survey • Longitudinal study of Triple
• Evaluation can increase delivery rates and reach.
g y pP cohort – tracking use of child protection, health, social and justice services
Strength
social and justice services.
• Custom-built scoring application to capture a range of client data.
Strength 4: Delivery management
Focus on selecting and training practitioners
Governance bodies need to extend their involvement to the management
R d ti f ti t l d li t
their involvement to the management of program delivery to clients.
Recommendations for active central delivery management:
• Memoranda of Understanding with service provider organisations
• Locking in delivery requirements
• Systems to track actual delivery.
Lessons for the futureThe emerging implementation science literature emphasises the importance of systems and practice transformation and is starting to explore alternatives to the training model in the science to serviceexplore alternatives to the training model in the science to service bridge.
In summary, large scale universal prevention and early intervention programs require:
i. Universal entry points soft, accessible, stigma-free
ii. Program promotion light, direct population-wide touch
iii. Data collection systems use input, output, outcome data -accountability
iv. Involvement in delivery management active and central delivery trackingdelivery tracking
v. Practitioner support early, consistent, collaborative.
Thank youa you
W t ?Want more?http://www.families.nsw.gov.au/assets/triplep_eval_report.pdf