Evidence-based public health

Evaluation of programs for health promotion is difficult, but they can be effective

I. Hancock L, Sanson-Fisher R W, Redman S et al. Community Action for Health Promotion: a review of methods and outcomes 1990-1995. Am J Prey Med 1997; 13:229-239 2. Watt D, Verman S, Flynn L. Wellness programs: a review of the evidence. Can Med

Assoc J 1998; 158:224-230

Background

There is increasing emphasis on the need to evaluate health promotion programs, but there is disagreement about the methods which should be used to evaluate such programs.

PAPER 1

Objective

To assess the effectiveness of community action for health promotion in the areas of cancer and heart disease.

Method

Critical literature review. Studies were evaluated according to a number of design criteria and only valid studies were included in the final analysis.

Literature review

Explicit strategy; defined databases, timespan ar, d search terms; 94 references.

Outcomes

1. Health outcomes; 2. Process measures.

Results

None of the 17 identified studies was found to meet the review criteria that the authors used. The main weaknesses identified in the review were related to sample size. The results from the most methodologically sound studies indicated no effect from community action for health promotion.

Authors' conclusions

The authors conclude that researchers of community-based health promotion programs need to adopt more sophisticated design and evaluation techniques before any conclusion can be reached about the effectiveness of such programs.

PAPER 2

Objective

To review studies of the association between

© llarcourt Brace and Company Lid 1998

wellness programs and improvements in quality of life.

Method

Systematic review with clear and explicit quality criteria stated in the paper. Of the 1082 articles found, only I 1 met all of the quality criteria.

Outcomes

1. Validated measures of quality of life; 2. Psychological status; 3. Locus of control.

Results

Of the original 1082 studies identified, 22 were initially selected, but 11 of these were discarded because they were descriptions or individual case studies. Significant problems were found in synthesizing the results of these studies. All studies produced evidence to support the use of the intervention they were studying and reported a high level of significance (P < 0.01). The results of the 11 individual papers are presented in a structured fashion, in part because the individuals covered by these wellness programs range from people with chronic health problems to healthy populations.

Authors' conclusions

The authors' general conclusion was that wellness-orientated interventions designed to promote health were effective, both for healthy people and those with chronic health problems. The authors conclude that a research strategy based on this systematic review would allow more consistency between studies and the use of recta- analysis.

:OMMENTAR~

PAPER I

The predominant emphasis in this paper is on the 'assessment of the evaluation strategies" used rather than on drawing inferences about the effects of these programs (which, though variously described as 'modest' or "disappo!nting,' are not actually set out systematically in the paper.) The authors appear torn between a professed expectation that such studies could be 'scientific' - in the sense of yielding generalizable results - and a recognition of the mult iple reasons why this may prove unattainable.

There is a basic diff iculty arising from the nature of the programs under review: if the form that 'community action for health promot ion' takes is genuinely al lowed to depend on local institutions, then:

I. The ' intervention' may cease to be objectively specifiable (in the way that a recommended daily dose of a drug is specifiable in the context of a clinical trial)

2. Effects may be contingent o n local circumstances in ways that escape the researchers' nets.

In each case, the generalizabil i ty of the findings is l imited and decisions to support such programs wil l continue to depend on informed judgment.

UK readers will still find plenty of ammunit ion to doubt the wisdom of their government's preference for programs of this kind, to be implemented, i t seems, w i thout even the recommended emphasis on rigorous evaluation)

PAPER 2

Watt et al show that a variety of structured interventions not involving drugs can improve

reported subjective well-being in a variety of study populations. Seven of the studies deemed eligible were RCTs. The fact that 'distant prayer' was shown to be effective in a 'double-bl ind RCT' wi l l lead some to suspect that sources of bias remained uncontrolled. Cognitive behavior therapy and exercise programs both make people feel better, and this is a major objective of health services.

Dr John Powles Institute o f Public Health

University of Cambridge, Cambridge, UK

Reference

1. Secretary of State for Health. Our healthier nation: a contract for health. London: The Stationery Office, 1998

68 EVIDENCE-BASED HEALTH POLICY AND MANAGEMENT SEPTEMBER 1998