the sleac method - fantaproject.org sleac method sleac is a low ... first stage sampling method....

The SLEAC Method

SLEAC is a low-resource method for classifying and estimating the coverage of selective feeding programs. It was designed to complement the SQUEAC method and is intended for use in programs delivering CMAM services over many service delivery units. Examples of such programs include:

• National or regional programs delivering CMAM services through health districts

• District programs delivering CMAM services through primary healthcare centres

SLEAC surveys classify coverage at the level of the service delivery unit. This will vary with the scale of the program. For example:

• In the case of a national or regional program delivering CMAM services through health districts (or the equivalent local administrative unit), the service delivery unit is the heath district and coverage is classified for an entire health district using a single SLEAC survey.

• In the case of a district program delivering CMAM services through primary healthcare centres, the service delivery unit is the primary healthcare centre and coverage is classified for each clinic’s catchment area using separate SLEAC surveys.

It is not usually sensible to treat units larger than a health district or units smaller than a clinic catchment area as service delivery units.

SLEAC can also be used to estimate coverage over wide areas. SLEAC has been used for regional and national coverage surveys. In these surveys coverage is usually classified and mapped at the district level and estimated at the regional and national levels.

SLEAC may be used in a number of ways:

• As a quick and simple way of investigating (classifying) coverage in service delivery units that returns limited information on barriers to service access and uptake.

• To identify service delivery units that are failing to achieve coverage targets. SQUEAC investigations undertaken in some or all of the failing service delivery units are then used to inform program reforms. SLEAC surveys are repeated (after a suitable interval) to confirm progress. This process is illustrated in Figure 67.

• To identify service delivery units that are successfully meeting coverage targets and service delivery units that are failing to meet coverage targets. SQUEAC investigations are then undertaken in one or more of the succeeding and one or more of the failing service delivery units so that factors influencing program success and failure can be identified and used to inform program reforms. SLEAC surveys are repeated (after a suitable interval) to confirm progress. This process is illustrated in Figure 68.

• To classify and map coverage over wide areas in district, regional, and national coverage surveys.

• To estimate coverage over wide-areas in district, regional, and national coverage surveys.

The design intention is that rapid and relatively cheap SLEAC surveys can be used to effectively target more intensive and more expensive SQUEAC investigations, which are then used to inform program reforms (Figure 67 and Figure 68). SLEAC surveys are then used to confirm progress.

SQUEAC/SLEAC Technical Reference 114

Figure 67. Using SLEAC and SQUEAC in failing service delivery units

StartSLEACsurveys

CoverageOK?

SQUEACinvestigation(s)

Reformprogram

Stop

Yes

No

Figure 68. Using SLEAC and SQUEAC in succeeding and failing service delivery units

Start SLEACsurveys

CoverageOK?


Reformprogram

Yes

No


Compare& contrast

SQUEAC and SLEAC are designed to complement each other:

SLEAC SQUEAC

SLEAC is a wide-area method that can be used to SQUEAC is a local method used to identify factors classify and map the coverage of CMAM service at influencing program success and failure at the local district, national, or regional levels. (i.e., district or clinic) level.

SLEAC provides a coarse overview of program coverage (i.e., coverage class) with only limited information on barriers.

SQUEAC provides a detailed view of program coverage and detailed information on barriers.


SLEAC may appear similar to the CSAS method. The key differences between the two methods are:

• SLEAC classifies coverage (e.g., as meeting or failing to meet a standard) at small scales, whereas CSAS estimates coverage at small scales (i.e., it returns a coverage proportion with a confidence interval).

• SLEAC can be used to map coverage classifications at the level of service delivery unit, whereas CSAS is intended to be used to map coverage in greater detail and usually within a service delivery unit (Figure 69).

• SLEAC can estimate coverage over several service delivery units, whereas CSAS usually estimates coverage for and within a single service delivery unit.

• A SLEAC survey will usually be very much quicker and very much cheaper than a CSAS survey of the same area.

Figure 69. The level of mapping available from SLEAC and CSAS methods

The SLEAC map shows coverage classified separately for 16 health districts in a single administrative region.The CSAS map shows coverage estimated for small areas within a single health district.

SLEACDistricts are classified as having low, moderate,

or high coverage. Regional or national mapping ofprogram coverage is possible.

CSASMapping of program coverage

within districts.

High

Moderate

Low

Shaded area represents period coverageon a scale of 0% to 100%.


Classifying Program Coverage

The SLEAC method classifies program coverage for a service delivery unit such as a health district.

A SLEAC survey does not provide an estimate of overall program coverage with a confidence interval or credible interval for a single service delivery unit. Instead, a SLEAC survey identifies the category of coverage (e.g., ‘low coverage’ or ‘high coverage’) that describes the coverage of the service delivery unit being assessed. The advantage of this approach is that relatively small sample sizes (e.g., n = 40) are required to make an accurate and reliable classification.

SLEAC can estimate coverage over several service delivery units. Coverage is still classified for individual service delivery units. Data from the individual service delivery units are combined and coverage for this wider area is estimated from this combined sample.

SLEAC Survey Sample Design

The sample design used in SLEAC surveys is the same as that used in SQUEAC likelihood surveys:

First stage sampling method. This is the sampling method that is used to select the villages to be sampled. CSAS assessments use the centric systematic area sampling or quadrat method to select villages to be sampled. A similar method could be used to select villages to be sampled for the SQUEAC likelihood survey. The number of quadrats drawn on the map may be much smaller than would be used for a CSAS assessment (this is the same as using larger quadrats). The villages to be sampled may be selected by their proximity to the centre of each quadrat as is done in a standard CSAS survey (Figure 58 and Figure 59). The number and size of quadrats should be selected so as to spread the sample of villages over the entire program area. Many small quadrats are better than few large quadrats. For example, the sample illustrated in Figure 59 (19 quadrats) spreads the sample more evenly and over more of the program catchment area than the sample illustrated in Figure 58 (8 quadrats). You should use as many quadrats as is feasible with the time and resources available for the survey. The CSAS/quadrat sampling method is appropriate for estimating coverage over a wide area such as a health district. Another useful approach is to stratify by clinic catchment area and select villages systematically from a complete list of villages sorted by clinic catchment area (Figure 60). This approach may be used with any areas (e.g., administrative areas) for which complete lists of villages are available. The first stage sampling method should be a spatial sampling method that yields a reasonably even spatial sample from the entire program catchment area. Cluster sampling using PPS, such as that used for SMART surveys, is not appropriate. The stratified approach outlined above and illustrated in Figure 60 provides a reasonably even spatial sample using village lists and does not require the use of maps. It is important to note that sampling should not stop when the survey has reached its required sample size. Sampling stops only after you have sampled all of the the selected villages.

A within-community sampling method. This will usually be an active and adaptive case-finding method or a house-to-house census sampling method (see Box 3, page 65). These methods find all, or nearly all, current and recovering SAM cases in a sampled village. Sampling should be exhaustive. This means that you stop sampling only when you are sure that you have found all cases in the community. Sampling should not stop when you have met a quota or when the wider survey has reached its required sample size.

This is a two-stage sample because a sample of villages in the survey area is taken first (Stage 1) and then a ‘census’ sample of current and recovering SAM cases is taken from each and every one of the selected villages (Stage 2).


The CSAS/quadrat approach is useful for a single survey. If you repeat the survey then the same villages will be sampled. This may cause the survey to overestimate coverage because we expect coverage to have been improved by case-finding and referral in the sampled villages. One way around this problem is to sample villages at random from each quadrat. This will yield independent samples at each survey round. Do not exclude previously sampled villages. This may cause the survey to underestimate coverage and you will eventually run out of villages to sample.

SLEAC Survey Sample SizeSLEAC uses a target sample size (n) which, together with prevalence and population estimates, is used to decide the number of villages (nvillages) that should be sampled to achieve the target sample size.

A target sample size of 40 (n = 40) cases from each service delivery unit in which coverage is to be classified is usually large enough for most SLEAC applications.

In some settings, it may be difficult or even impossible to find 40 (n = 40) cases. This will be the case if service delivery units are small and/or the prevalence of SAM is low. In these situations, it is possible to use a smaller target sample size without increasing error. Table 5 shows target sample sizes that may be used when the total number of cases in a service delivery unit is likely to be small. If, for example, the total number of cases in a service delivery unit is estimated to be about 60 cases then a target sample size of 25 cases may be used.

Table 5. Target sample sizes for 50% and 70% coverage standards for use when surveying small service delivery units and/or the prevalence of SAM is low

Target sample size for …

Total number ofcases in the service

delivery unit*50% standard

70% standard or 30%/70% class thresholds

500 37 33

250 35 32

125 31 29

100 29 26

80 27 26

60 25 25

50 23 22

40 21 19

30 17 18

20 15 15* This is an estimate of the number of SAM cases in a service delivery unit at the time of the survey:

⌊Populationall ages ×percentage of population 6– 59months

100× SAM prevalence

100 ⌋The target sample size (n), together with estimates of the prevalence of SAM in the survey area and population data, is used to calculate the number of villages (nvillages) that will need to be sampled to achieve the target sample size:

nvillages = ⌈ npercentage of population

average village population × 6 -59 months SAM prevalenceall ages ×

100 100 ⌉SQUEAC/SLEAC Technical Reference 118

SAM prevalence refers to the average SAM prevalence in the catchment area of the service delivery unit. It is unlikely that SAM prevalence will be known or known with good precision. SAM prevalence estimates may be available from previous nutritional anthropometry surveys (e.g., SMART surveys). SAM prevalence varies throughout the year (e.g., prevalence is usually higher before harvests than after harvests). This means that you should use the results from a nutritional anthropometry survey undertaken at the same time of year as the current SLEAC assessment. It is better to use a low rather than a high estimate of SAM prevalence for this sample size calculation. A value midway between the point estimate and the lower 95% confidence limit for SAM prevalence could be used. For example, if the prevalence of SAM is estimated as 1.2% (95% CI = 0.6% – 2.6%) then a suitable low estimate would be:

1.2 − 0.6Prevalence = 1.2 − = 0.9%

2

Using a low estimate helps ensure that the survey will achieve the target sample size.

Note that prevalence here is the estimated prevalence of the program’s admitting case definition. This will usually not be the weight-for-height based ‘headline’ prevalence estimate reported by a SMART survey. The required estimate will usually be found in the needs assessment section of a SMART survey report.

If you do not have nutritional anthropometry survey results from the same time of year as the current SQUEAC assessment then you should use results from the most recent nutritional anthropometry survey and adjust them using, for example, seasonal calendars of human disease (Figure 6, Figure 11, and Figure 12), calendars of food availability (Figure 6, Figure 11, and Figure 12), agricultural calendars (Figure 6, Figure 11, and Figure 19), long-term admissions data from nutrition programs (Figure 8), and long-term returns from growth monitoring programs.

The formula for the calculation of the minimum number of villages that need to be sampled to achieve the required sample size shown above assumes that the case-finding method being used will find all, or nearly all, current and recovering SAM cases in sampled villages. If you are not sure of this then you should sample a larger number of villages. You should monitor the number of cases that are found during the survey and be prepared to increase the number of villages that will be sampled if many fewer cases than expected are being found.

Once these decisions and calculations have been made, sampling locations can be identified and the survey undertaken. A standard questionnaire, such as that shown in Box 2 (page 49), should be applied to carers of non-covered cases found by the survey. Data collected using the standard questionnaire (Box 2) can be presented using a Pareto chart (a bar chart in which the bars are ordered by frequency) similar to those shown in Figure 2, Figure 45, and Figure 46).


Here is an example of the calculations required to decide the number of villages (nvillages) to sample:

Target sample size. A target sample size of n = 40 cases was selected. This is the standard sample size for a SLEAC survey.

Number of villages to be sampled. The following information was used to calculate the number of villages (nvillages) to be sampled:

Target sample size : 40

Average village population (all ages) : 475

Prevalence of SAM : 1.5%

Percentage of children aged between 6 and 59 months : 18%

The percentage of children aged between 6 and 59 months is usually assumed to be about 20% in developing countries. You should use 20% unless you have better information from, for example, a recent census or population survey.

Using this information, the number of villages to be sampled was calculated to be:

nvillages = ⌈ 40 =18 1.5475 × ×100 100 ⌉ 32

When area sampling (see Figure 58 and Figure 59) is used, then the villages to be sampled are distributed evenly between the areas. For example, if a CSAS/quadrat sample with eight quadrats such as that shown in Figure 58 is used and 32 villages are to be sampled then:

⌈ 32 ⌉= 48

villages will need to be sampled from each quadrat.

In the case of a CSAS sample (e.g., Figure 58 and Figure 59), villages to be sampled are selected by their proximity to the centre of each quadrat (Figure 61). This selects clusters of villages and reduces the travel time between villages selected to be sampled. This allows more villages to be sampled by a survey team in a day.

A CSAS sample requires a map. If a map is not available then an alternative spatial stratification method may be used. Figure 60 shows a sample stratified by clinic catchment area. Any areal unit or subdivision for which complete lists of villages are available (e.g., counties, vice-counties, chiefdoms, electoral divisions) may be used. Figure 62, for example, illustrates the process of taking a spatially stratified systematic sample from a list of villages sorted by chiefdom. This type of sample also spreads the sample over the entire survey area.

Box 2 (page 49) shows an example of a simple structured interview questionnaire that may be applied to carers of non-covered cases found during the survey. The questionnaire shown in Box 2 yields qualitative data (i.e., questions regarding the how? and why? of decision making in carers of non-covered cases) that can be analysed using simple quantitative techniques as in Figure 2, Figure 45, and Figure 46).


Classifying Coverage in Individual Service Delivery Units

SLEAC uses the same simplified LQAS classification technique that is used in SQUEAC small-area surveys. The differences between how the simplified LQAS classification technique is used in SQUEAC and SLEAC are:

• The SLEAC survey sample is designed to represent the entire program area.

• A target sample size for SLEAC surveys is decided in advance of data collection.

• SLEAC surveys may classify coverage into three or more classes.

Analysis of data using the simplified LQAS classification technique involves examining the number of cases found in the survey sample (n) and the number of covered cases found:

• If the number of covered cases found exceeds a threshold value (d) then coverage is classified as being satisfactory.

• If the number of covered cases found does not exceed this threshold value (d) then coverage is classified as being unsatisfactory.

The threshold value (d) depends on the number of cases found (n) and the standard (p) against which coverage is being evaluated. A specific combination of n and d is called a sampling plan.

The following rule-of-thumb formula may be used to calculate a suitable threshold value (d) for any coverage proportion (p) and any sample size (n):

d = ⌊ pn ×

100 ⌋For example, with a sample size of n = 40 and a coverage proportion (p) of 70% an appropriate value for d would be:

d = ⌊ pn × ⌋= ⌊ 70

40 × ⌋= ⌊ 40 × 0.7 ⌋ = 28100 100

It is unlikely that a SLEAC survey will return the target sample size (n) exactly. If a survey does not return the target sample size (n) exactly then the classification threshold value (d) should be recalculated using the achieved sample size. For example:

Target sample size : 40

Achieved sample size : 43

Standard : 70%

70d : ⌊43 ×

100 ⌋= 30

Coverage is classified using the same technique as is used for SQUEAC small-area surveys. For example:

n : 43

d : 30

Covered cases found : 34

Coverage classification : Satisfactory (since 34 > 30)


Extending the Classification Method to Yield Finer Classifications

The simplified LQAS classification technique provides binary or two-tier classifications.

The method may be extended to provide more granular classifications.

Three classes are sufficient for most SLEAC applications. A three-tier classification method is particularly useful for identifying very high coverage service delivery units and very low coverage service delivery units for inclusion in subsequent SQUEAC investigations when using the SLEAC/SQUEAC strategy illustrated in Figure 68.

Three-tier classifications require two sampling plans/decision rules. These are created using the rule-of-thumb formula presented earlier.

For three-tier classifications there are two coverage proportions:

p1 : The upper limit of the ‘low coverage’ tier or class

p2 : The lower limit of the ‘high coverage’ tier or class

The ‘moderate coverage’ class runs from p1 2

p1

p2

to p . For example:


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

High coverageLow coverage Moderate coverage

Program coverage

Two classification thresholds (d1 and d2) are used and are calculated as:

pd 1 p2

1 =⌊ n ×100 ⌋ d 2 = ⌊ n ×

100 ⌋Classifications are made using the algorithm illustrated in Figure 70.

Figure 70. Algorithm for a three-class simplified LQAS classifier

Sample

Classifyas high

coverage

Classifyas moderate

coverage

Classifyas low

coverage

Number ofcovered casesexceeds d

2?

Yes

No

Yes

NoNumber of

covered casesexceeds d

1?

This three-tier classification works well with small sample sizes (e.g., n = 40) provided that the difference between p1 and p2 is greater than or equal to about 20 percentage points.

Here is an example of the calculations required:

Sample size (n) : 40

p1 : 30%

p2 : 70%

d1 : ⌊ pn × 1 ⌋ 30=⌊ 40 × =

100 100 ⌋ 12

d2 : ⌊ pn × 2

100 ⌋= ⌊ 7040 ×

100 ⌋= 28

Classifications are made using the algorithm illustrated in Figure 70. For example, using the calculations:

Sample size (n) : 40

p1 : 30%

p2 : 70%

d1 : ⌊ pn × 1

100 ⌋= ⌊ 3040 ×

100 ⌋= 12

d2 : ⌊ pn × 2

100 ⌋ 70=⌊ 40 ×100 ⌋= 28

the following classifications are made:

Number of Classificationcovered cases

1, 2, …, 12 LOW (i.e., < 30%) coverage

13, 14, …, 28 MODERATE (i.e., between 30% and 70%) coverage

29, 30, …, 40 HIGH (i.e., ≥ 70%) coverage

Figure 71 shows a nomogram that can be used to find appropriate values for d1 and d2 given n, p1, and p2.


Figure 71. Simplified LQAS nomogram for finding appropriate values for d1 and d2 given n, p1, and p2


0 5 10 15 20 25 30 35 40 4530

40

50

d (round down to nearest whole number)

Sam

ple

size

(n

)p = 10% p = 20% p = 30% p = 40% p = 50% p = 60% p = 70% p = 80% p = 90%

Example showing d = 27 when n = 39 and p = 70%

If a survey does not return the target sample size (n) exactly then the classification thresholds (d1 and d2) should be calculated using the achieved sample size and classifications made using the algorithm illustrated in Figure 70. For example, a survey classifying coverage in individual clinic catchment areas using a target sample size of 40 (n = 40) cases for each catchment area and the class boundaries p1 = 30% and p2 = 70% returned the following data:

Cliniccatchment area

Samplesize

*d1*d2

Number ofcovered cases

Classification

Chawama 38 11 26 29 High

Matero 32 9 22 18 Moderate

Makeni 43 12 30 36 High

Chipata 35 10 24 15 Moderate

Ngombe 42 12 29 14 Moderate

Kalingalinga 37 11 25 10 Low

Mtendere 39 11 27 5 Low

Kanyama 42 12 29 23 Moderate

All 308 92 215 150 Moderate* d1 and d2 calculated after data collection using achieved sample sizes.

In this example, the target sample size was applied to each of the clinic catchment areas separately. This allows coverage classifications to be made for individual clinic catchment areas. This approach enables the identification of low coverage and high coverage service delivery units (clinics in this example) for subsequent SQUEAC investigations when using the SLEAC/SQUEAC strategies illustrated in Figure 67 and Figure 68.The example given here classifies coverage in clinic catchment areas in a single health district. These coverage classifications could be presented as a map:


High

Moderate

Low

10 km

NgombeKanyama

Mtendere

Kalingalinga

ChawamaMakeni

Matero

Chipata

A similar approach is applied to national or regional coverage surveys. In the case of national and regional coverage surveys, the service delivery units assessed by SLEAC should not be larger than individual health districts.

National and regional coverage surveys using SLEAC are stratified sample surveys in which strata are defined by health districts and sampled exhaustively (i.e., a SLEAC survey is undertaken in each and every health district in the nation). Such a survey will produce classifications of program coverage in each and every health district that can be mapped. Regional and national estimates of program coverage may also be produced (see below).

An alternative approach for identifying very high coverage service delivery units and very low coverage service delivery units for inclusion in subsequent SQUEAC investigations when using the SLEAC/SQUEAC strategies illustrated in Figure 67 and Figure 68 is to use coarse estimates of coverage:

Number of covered casesCoverage =

Sample size

in each surveyed area and pick the extreme values for further investigation:

Cliniccatchment

area

Samplesize

Number ofcovered cases

Coverage (%)Selected

for strategy inFigure ...

Chawama 38 29 2938

× 100= 76 % Figure 68*

Matero 32 181832

× 100= 56%

Makeni 43 363643

× 100= 84 % Figure 68

Chipata 35 151535

× 100= 43%

Ngombe 42 141442

× 100= 33%

Kalingalinga 37 101037

× 100= 27% Figure 67* and Figure 68*

Mtendere 39 5539

× 100= 13% Figure 67 and Figure 68

Kanyama 42 232342

× 100= 55%

* More than one ‘best’ and more than one ‘worst’ may be selected if there are a large number of areasand funding is available for additional SQUEAC investigations.

The advantage of using this approach is that there is no need to define high and low coverage categories in advance. This approach is also useful when the three-class method ‘fails’ and, for example, classifies all service delivery units as having low coverage.

Note that the coverage estimate is used solely for identifying the probable best and probable worst performing service delivery units.


Estimating Coverage over Wide Areas

It is also possible to estimate coverage over several service delivery units.

The number of SAM cases will vary between service delivery units in the program area. This means that the results from any one service delivery unit should be weighted by the number of cases in that service delivery unit.

The number of cases in each service delivery is unknown but can be estimated as:

⌊ percentage of populationN SAM prevalence= population of service delivery unit all ages ×

6- 59months ×100 100 ⌋

The percentage of children aged between 6 and 59 months is usually assumed to be about 20% in developing countries. You should use 20% unless you have better information from, for example, a recent census or population survey.

If SAM prevalence is not known then a sensible guess should be used.

The weighting factor for each survey is:

Nw =

∑N

where:

N : Estimated number of cases in a surveyed service delivery unit

∑ N : The sum of N over all surveyed service delivery units

The weighting factors for each survey (w) is based on estimates of the number of cases in each service delivery unit (N). These estimates are based on estimates of population size, population structure, and the prevalence of SAM:


N =⌊ population of service delivery unit all ages ×percentage of population6 -59 months

100× SAM prevalence

100 ⌋

Population size Population structure Prevalence of SAM

The weighting factors should be as accurate as possible and be local to each survey. This means that, whenever possible, accurate and local estimates should be used to calculate the weighting factor (w).

Failure to use local estimates may cause too much or too little weight to be given to particular surveys. This may result in biased (i.e., inaccurate) wide area estimates of coverage. It may also make coverage appear to be patchy when it is, in fact, even (or vice versa).

Populations can change rapidly due to, for example, crisis displacement and population estimates from, for example, a past census may need to be adjusted.

The prevalence of SAM varies over both time and space. For example, neighbouring populations may have different prevalences of SAM due to differing food-economies, childcare practices, or patterns of disease. You should take care to use the appropriate local SAM prevalence estimate that is

available to you. It is almost never appropriate to use regional or national estimates of SAM prevalence from DHS or Multiple Indicator Cluster Survey (MICS) surveys.

Point coverage is estimated as:

Coverage =∑[ cw ×

n ]where:

Nw : weighting factor w =

∑ for each survey (see above)N

c : number of covered cases found in each survey

n : number of current cases attending the program plus the number of current casesnot attending the program found in each survey

Period coverage may be estimated using the same formula with:

c : number of current and recovering cases attending the program found in each survey

n : number of current and recovering cases attending the program plus the number ofcurrent cases not attending the program found in each survey

The example data are for clinics within a single health district. The same method is used for national or regional coverage surveys that sample all districts in a nation (national coverage survey) or all districts within a region (regional coverage survey).

Applying this method to the example clinic-level data gives:

Population data Survey data Result

6 – 59 SAM N c cClinic Pop. w=N n c w ×

months* prevalence* ∑N n n

115 29Chawama 28750 18.4% 2.2 % ⌊28750×0.184×0.022⌋=115 =0.13 38 29 =0.76 0.13×0.76=0.0988900 38

90 18Matero 22456 18.4% 2.2 % ⌊22456×0.184×0.022⌋=90 =0.10 32 18 =0.56 0.10×0.56=0.0560900 32

121 36Makeni 30050 18.4% 2.2 % ⌊30050×0.184×0.022⌋=121 =0.13 43 36 =0.84 0.13×0.84=0.1092900 43

114 15Chipata 28308 18.4% 2.2 % ⌊28308×0.184×0.022⌋=114 =0.13 35 15 =0.43 0.13×0.43=0.0559900 35

98 14Ngombe 24335 18.4% 2.2 % ⌊24335×0.184×0.022⌋=98 =0.11 42 14 =0.33 0.11×0.33=0.0363900 42

104 10Kalingalinga 25737 18.4% 2.2 % ⌊25737×0.184×0.022⌋=104 =0.12 37 10 =0.27 0.12×0.27=0.0324900 37

132 5Mtendere 32767 18.4% 2.2 % ⌊32767×0.184×0.022⌋=132 =0.15 39 5 =0.13 0.15×0.13=0.0195900 39

125 23Kanyama 31043 18.4% 2.2 % ⌊31043×0.184×0.022⌋=125 =0.14 42 23 =0.55 0.14×0.55=0.0770900 42

SUM 18.4 % 2.2 % 899 1.00 308 150 0.4851

* These data were available only at the district level and are the same for each clinic. If prevalence data for each clinic catchment area were available then N (the number of SAM cases) and w (the weighting factor) would have been calculated using the prevalence specific to each clinic catchment area.

The estimated coverage in the area served by the eight clinics is 0.4851 or 48.51%.


A 95% confidence interval on the estimated coverage can be calculated using the following formula:

√∑ w 2 c× ×n

95%CI = Coverage ±1.96 ×( c1 −

n )n

Applying this formula to the example data gives:

2 c cw ×

2 c c ×(1 − )Clinic n c w w 1− n nn n

n

29 0.0169×0.76×0.24Chawama 38 29 0.13 0.0169 =0.76 0.24 =0.00008112

38 38

18 0.0100×0.56×0.44Matero 32 18 0.10 0.0100 =0.56 0.44 =0.00007700

32 32

36 0.0169×0.84×0.16Makeni 43 36 0.13 0.0169 =0.84 0.16 =0.00005282

43 43

15 0.0169×0.43×0.57Chipata 35 15 0.13 0.0169 =0.43 0.57 =0.00011835

35 35

14 0.0121×0.33×0.67Ngombe 42 14 0.11 0.0121 =0.33 0.67 =0.00006370

42 42

10 0.0144×0.27×0.73Kalingalinga 37 10 0.12 0.0144 =0.27 0.73 =0.00007671

37 37

5 0.0225×0.13×0.87Mtendere 39 5 0.15 0.0225 =0.13 0.87 =0.00006525

39 39

23 0.0196×0.55×0.45Kanyama 42 23 0.14 0.0196 =0.55 0.45 =0.00011550

42 42

SUM 308 150 0.00065045

The 95% confidence interval is:

95%CI = 0.4851 ±1.96 × √0.00065405 = { 43.51% , 53.51%}

It is usually only sensible to report an overall coverage estimate if:

• The overall sample size is about 96 (or larger). This sample size is usually sufficient for a 95% confidence interval of ± 10 percentage points or better.

• Coverage is not patchy (i.e., coverage is broadly similar in each of the areas surveyed).

The patchiness of coverage can be assessed ‘by eye’ or using a chi-square test.

The chi-square test is a statistical hypothesis test. Statistical hypothesis tests such as the chi-square test rely on a null hypothesis. The null hypothesis states that nothing interesting is happening in the data other than random variation (e.g., coverage is not patchy). The null hypothesis is paired with an alternative hypothesis that states that something interesting or systematic is happening in the data (e.g., coverage is patchy).

Statistical hypothesis testing involves comparing what we would expect the data to look like if the null hypothesis were true with the collected or observed data. If the expected and observed data are very different from each other then we reject the null hypotheses and accept the alternative hypothesis.


If we are testing whether coverage is patchy then:

• The null hypothesis is that coverage is not patchy.

• The alternative hypothesis is that coverage is patchy.

The simplest illustration of using a chi-square test is to use it to compare the coverage of two service delivery units.

Consider the following data:

Observed data

Clinic n O

Chawama 38 29

Kalingalinga 37 10

The null hypothesis is that coverage is not patchy. Another way of saying this is that coverage is uniform (i.e., the same) in both service delivery units. If the null hypothesis were true then we would expect the data to look like this:

Observed data Expected data*

Clinic n O E

39Chawama 38 29 ×38=19.7675

39Kalingalinga 37 10 ×37=19.2475

SUM 75 39 39* These are the numbers we would expect to see if coverage in each

clinic catchment area were the same as the average coverageacross all clinic catchment areas.

These expected values (E) are the values we would expect to see if the null hypothesis were true.

The expected values are compared with the observed values (O) by subtracting them from the observed values:

Observed data Expected data

Clinic n O E O−E

39Chawama 38 29 ×38=19.76 29−19.76=+9.2475

39Kalingalinga 37 10 ×37=19.24 10−19.24=−9.2475

SUM 75 39 39 0

The positive and negative differences cancel each other out. We square each difference to make them positive numbers:


Clinic n O E O−E (O−E )2

39Chawama 38 29 ×38=19.76 29−19.76=+9.24 85.3875

39Kalingalinga 37 10 ×37=19.24 10−19.24=−9.24 85.3875

SUM 75 39 39 0


Before dividing them by the expected values:


OClinic n O E O−E (O E )2 ( −E )2−E

39 85.38Chawama 38 29 ×38=19.76 29−19.76=+9.24 85.38 =4.3275 19.76

39 85.38Kalingalinga 37 10 ×37=19.24 10−19.24=−9.24 85.38 =4.4475 19.24

SUM 75 39 39 0 8.76

The sum of these two numbers (8.76 in this example) is a measure of how much the observed data differ from the expected values under the null hypotheses and is called the chi-square test statistic.

Under the null hypothesis there is a fixed probability (called the p-value or just p) of obtaining a particular value for the chi-square test statistic:

• If the probability of obtaining a particular chi-square test statistic under the null hypothesis is large then the probability that the null hypothesis is true is also large. In this case, we would accept the null hypothesis (i.e., coverage is uniform) as being true.

• If the probability of obtaining a particular chi-square test statistic under the null hypothesis is small then the probability that the null hypothesis is true is also small. In this case, we would reject the null hypothesis and accept the alternative hypothesis (i.e., coverage is patchy) as being true.

It is common practice to define large as p ≥ 0.05 and to define small as p < 0.05.

The value of the chi-square test statistic at p = 0.05 is known as the critical value. The value of the chi-square test statistic is compared to the critical value. If the chi-square test statistic is greater than the critical value then p < 0.05 and the null hypothesis is rejected.

The critical value of the chi-square test statistic changes with the number of surveys used to calculate the chi-square test statistic and is shown in Table 6.

There are two surveys in this example. The critical value of the chi-square test statistic for two surveys is 3.84 (see Table 6). Since 8.76 is greater than 3.84, we reject the null hypothesis and conclude that coverage is patchy.


Table 6. Critical values of the chi-square test statistic

Numberof surveys

Criticalvalue*

Numberof surveys

Criticalvalue*

Numberof surveys

Criticalvalue*

1 NA 13 21.03 25 36.42

2 3.84 14 22.36 30 42.56

3 5.99 15 23.68 40 54.57

4 7.81 16 25.00 50 66.34

5 9.49 17 26.30 60 77.93

6 11.07 18 27.59 70 89.39

7 12.59 19 28.87 80 100.75

8 14.07 20 30.14 90 112.02

9 15.51 21 31.41 100 123.23

10 16.92 22 32.67

11 18.31 23 33.92

12 19.68 24 35.17* Corresponds to p = 0.05 on a chi-square statistic with N – 1 degrees of freedom

The chi-square test statistic can be used to assess patchiness over any number of service delivery units. The formula to calculate the chi-square test statistic is:

χ2=∑ (O − E )2

Ewhere:

O : Number of covered cases observed in each surveyed service delivery unit.

E : Number of covered cases expected in each surveyed service delivery unit ifcoverage is not patchy. You will need to calculate this.

The chi-square test presented here evaluates how much the observed numbers deviate from the numbers expected if coverage in each service delivery unit were the same as the overall coverage estimate.

If coverage is patchy then this should be noted in any report of the overall coverage estimate.


The table below applies the chi-square test to the example data:

Cliniccatchment

area

Samplesize

O * E ** 0 − E 2 0 − E 2

E

Chawama 38 29 38 ×150308 = 18.51 29 − 18.512 = 110.04

110.0418.51 = 5.94

Matero 32 18 32 ×150308 = 15.58 18 − 15.582 = 5.86

5.8615.58 = 0.38

Makeni 43 36 43 ×150308 = 20.94 36 − 20.942 = 226.80

226.8020.94 = 10.83

Chipata 35 15 35 ×150308 = 17.05 (15 − 17.05)2 = 4.20

4.2017.05 = 0.25

Ngombe 42 14 42 ×150308 = 20.45 14 − 20.452 = 41.60

41.6020.45 = 2.03

Kalingalinga 37 10 37 ×150308 = 18.02 10 − 18.022 = 64.32

64.3218.02 = 3.57

Mtendere 39 5 39 ×150308 = 18.99 5 − 18.992 = 195.72

195.7218.99 = 10.31

Kanyama 42 23 42 ×150308 = 20.45 23 − 20.452 = 6.50

6.5020.45 = 0.32

SUM 308 150*** 150*** 2= 33.63

**

* The number of covered cases observed in each survey

The number of covered cases expected in each survey if coverage is *** These columns should have the same total

not patchy

The chi-square test statistic for the example data is 33.63.

The value of the chi-square test statistic is compared to a critical value. If the chi-square statistic is greater than the critical value then the coverage is patchy and it is more meaningful to report the disaggregated results than an overall coverage estimate. If you do report an overall coverage estimate then you should also report that the coverage was found to be patchy.

The critical value of the chi-square test statistic changes with the number of surveys used to calculate the chi-square test statistic and is shown in Table 6 (page 130). There are eight surveys in the example data. The critical value of the chi-square statistic for eight surveys is 14.07 (see Table 6). Since 33.63 is greater than 14.07, we conclude that coverage is patchy and it is better to report disaggregated results than an overall coverage or to report the overall coverage estimate and report that the coverage was found to be patchy.

Conclusions

SLEAC provides a quick and simple method for classifying coverage in program service delivery units and provides limited data (i.e., reasons for non-attendance collected from a single informant type using a single method with a small sample size) on barriers to service uptake and analysis. SLEAC offers program managers a method of targeting more intensive and expensive SQUEAC investigations when gathering evidence to inform program reforms. SLEAC also offers regional and national program managers a reasonably quick and simple method for mapping coverage over very wide areas.


the sleac method - fantaproject.org sleac method sleac is a low ... first stage sampling method....

Documents