claire galligan - caap 2015 evaluation report
TRANSCRIPT
CELEBRATING ACHIEVEMENT NATIONAL COMMUNITY ACTION ON
ALCOHOL PILOT PROJECT 2015 CAAP©
COMMUNITY ACTION ON ALCOHOL PILOT PROJECT
EVALUATION REPORTCLAIRE GALLIGAN
SOCIAL RESEARCH CONSULTANT
COMMUNITY MOBILISATION ON ALCOHOLEVALUATION REPORT
CLAIRE GALLIGAN BA, MA, PDE, PGCERT
COMMUNITY ACTION ON ALCOHOL, PROJECT GOALS • To introduce a model of community mobilisation• To build awareness of alcohol related harm• To raise awareness of community mobilisation measures as
part of the National Substance Misuse Strategy• To promote community engagement• To ensure evaluation measures are built into the
development of plans
GOALS OF THE EVALUATION
To carry out a process evaluation to explore …• The Quality of Training • Assess the knowledge/awareness of alcohol related harms• Identify the number of action plans developed• Assess how the project can contribute to current research
and development
OVERVIEW OF APPROACH
Literature review
Early focus group with trainees
Overview of documents
Observation of training sessions
Observation of facilitated
sessions
METHODOLOGY
• Process Evaluation - concerned with HOW a programme is actually delivered• Takes into account, the process and the context in which it took place• Qualitative evaluation – provides a rich description • Range of methods used, including observation and focus groups• Analysis of data was carried out in a systematic way, to lessen the bias of
the researcher• Questions designed around the goals of the evaluation
LITERATURE REVIEW• Draws from peer reviewed studies about community
mobilisation and good practice in adult/public health education
Formation of a community coalition,
involving key stakeholders
Training in areas of alcohol related
harm and evidence based
measures
Local research to
inform actions
Identifying approach to
address harms identified (Includes a media strategy)
Implementing plan (including
policy change)and evaluating progress
KEY FINDINGS OF LITERATURE REVIEW
• Community mobilisation has been effective in reducing alcohol related harms• While projects adopt a similar approach, the processes used can differ• Limited data about training for community mobilisation on alcohol, however one model shows that
training is linked to more effective community coalitions and fidelity to science based approaches• Aspects of effective models include: engaging a wide range of stakeholders, good leadership
capacity, a functioning community coalition, availability of resources, full time coordinator, adoption to science based approaches and using the media.
• Community mobilisation can take place in communities where there is resistance to seeing alcohol as a problem
• The timeframe of a community mobilisation process can be around 5 years
KEY FINDINGS OF LIT REVIEW (CONTD.)
• Key strategies employed by coalitions include grass roots organising, building leadership capacity in the community, working to enforce existing laws, direct action, negotiating with store owners and education/training (including training community members to conduct research)
• Community mobilisation is most effective in a supported policy environment• Community mobilisation is measured using already available statistics, utilising control
communities, proxy buyers and self assessment measures • Training is delivered onsite with leadership boards • Technical assistance from university staff, to carry out research and analysis is also seen
as a supportive factor
LIT REVIEW: QUALITY TRAINING
• Adults need to know why they are learning• Adults are motivated to learn by the need to solve problems• Adults previous experience must be respected and built upon• Learning approaches should match adults background and diversity• Adults need to be actively involved in the learning process• DEWF standards are similar to above, but also need to
contextualise programme within current drug strategies
PROJECT NARRATIVE
Taskforces invited to apply. Five selected
Selected
Taskforces
informed of projec
t requirement
s
Taskforces establish
Alcohol
Sub Committe
es
Representatives take part in 5
formal
training
days
Taskforces
facilitated to
develop
alcohol
action plans
PRIMARY RESEARCH FINDINGS: TRAINING
• Training objectives mirror key priorities outlined in the literature review• Training materials are organised, easy to follow and link closely with session
objectives • Training explores participants ‘socialisation’ in relation to alcohol• Training methodologies mirror best practice in adult education (re drug
prevention)• Training perceived as very high quality by all trainees• Some trainees have prior knowledge of some training objectives, however it gives
them confidence and validates them in their approach
PRIMARY RESEARCH FINDINGS: LEARNING OUTCOMES (2)• Trainees increased their knowledge of alcohol related harm, the policy
environment and community mobilisation approaches• Structure/location of training was difficult for groups outside of Dublin • Not always easy for trainees to ‘feed back’ to their sub committees • All trainees report the positive impact of networking and the project
having a national focus• Facilitated sessions viewed positively by all groups, in particular, the
knowledge and expertise of trainer
PRIMARY RESEARCH FINDINGS: IMPACT ON WORK PRACTICES • Questionnaire data shows that training was impacting on work practices• The project puts the focus on alcohol for all participant groups• Challenges groups to adopt a ‘public health model’ approach to drug use• Trainees apply learning to other aspects of their work • Stakeholder attendance at training may impact on the adoption of work practices.
Higher fidelity to approach where larger range of stakeholders attend training.• Not all groups establish a leadership group with a range of stakeholders • All projects report that alcohol work is a strain on their resources and takes away from
other work
COMMUNITY ACTION PLANS TO ADDRESS ALCOHOL MISUSE • Four out of five Task Force Groups completed action plans
that included monitoring, review and evaluation measures • Stakeholder engagement in the planning process varied
between groups
ENABLING FACTORS
• High quality training increases knowledge of alcohol related harms • Facilitated sessions help groups to apply learning and adopt
lessons• Participants gain from the networking experience • Knowledge and expertise of trainer• High and consistent level of attendance • Full participation and enjoyment
CRITICAL BARRIERS AND STRATEGIES TO ADDRESS THEM• Additional supports given by trainer, above and beyond the project brief • Training gaps for some participants, are addressed through additional inputs
by trainer• Some projects do not engage a range of stakeholders, despite supports given• Location and time needed seen as a barrier by some participants • Resources to implement seen as a barrier by all participants • Project is very early in the mobilisation process, projects with limited growth
to date still intend to adopt lessons
DETERMINANTS OF SUSTAINABILITY AND TRANSFERABILITY • Training has supported the process of ‘embedding’ alcohol work within the taskforces• Ongoing support for groups following the formal training process has also supported
sustainability • Limited resources may affect sustainability in the longer term• The wider policy environment does not always support the proposed changes to
address alcohol harm at a community level• Some boards did not engage a range of stakeholders .. Indicates limited community
ownership, may impact on sustainability • Engagement with university researchers may also support sustainability
FURTHER RESEARCH
• Limited research available on community mobilisation processes and very little on community mobilisation on alcohol in Ireland• Factors that lead to effective change are wider than training, further
research may explore this• Evidence from this project shows that community mobilisation can
address policy implementation processes – this could be further explored • Research to explore community mobilisation in an Irish context, including
the measurement of risk and protective factors could be undertaken.
RECOMMENDATIONS: (1) FACTORS FOR STRATEGIC DEVELOPMENT
• Implementation of the Public Health (Alcohol) Bill can potentially support community mobilisation measures and vice versa.• Build supports to ensure fidelity to evidence based
approaches in local action planning• Encourage cooperation between practitioners and
researchers (e.g. through universities) in the public health field
RECOMMENDATIONS: (2) INVESTING IN COMMUNITY ACTION AT A LOCAL LEVEL• Establish community coalitions and engage them in
induction/training early in the process• There should be full participation of leaders in the training
process, along with the other stakeholders• Managing resources/funding to enable community
mobilisation on alcohol
RECOMMENDATIONS: (3) DELIVERY OF TRAINING • Materials were of a very high standard overall – however, increase
materials on evidence based community mobilisation processes• Deliver the bulk of training/facilitation on site with groups to enhance
collective leadership capacity and tailor it to group needs• Retain a networking element to the project• Extend the time available to groups to get established and develop plans
(18 months)• Maintain technical support for groups after plans have been completed.
FINALLY …
• Thanks to Anne Timoney Meehan, the steering committee and all of the participating Task Forces and their stakeholders who helped to inform this evaluation.
• Thank you all for listening!
CELEBRATING ACHIEVEMENT NATIONAL COMMUNITY ACTION ON
ALCOHOL PILOT PROJECT 2015 CAAP©
THE COMMUNITY ACTION ON ALCOHOL PROJECT WAS DESIGNED, DEVELOPED AND DELIVERED BY
ANNE TIMONY MEEHAN, ALCOHOL FORUM