Does it matter what estimation method I use to provide

small area populations at risk in standardised mortality

ratios?

CCSR Seminar: 16th December 2003

Paul Norman

Context• Rates of health may need to be calculated for small geographical areas

• Census years we have age-sex population counts for a range of geographical areas, but outside census years …

• Annual age-sex disaggregated mid-year estimates only available down to local authority level

• Various small area population estimation methods commonly used

• Studies have shown variation in population sizes & age structures

• Lunn et al. (1998)

• Middleton (1996)

• Simpson et al. (1996 and 1997)

• Rees (1994)

• Differently estimated small area ‘populations at risk’ may lead to different SMRs if different size &/or age-sex structure

Indirect Standardised Mortality Ratio (SMR)

SMR = Observed mortality events

Expected mortality events

SMR = 100 x

Deaths in a location of interest

Deaths in a standard area population

Population in the standard area

Population in the

location of interestX

Observed

Expected

Data sources for indirect SMRs at ward level

SMR = 100 x Deaths in a location of interest

Mortality data for the ward (VS4)

Mortality data at national level

Population estimate at national level

Population estimate for the ward

Deaths in a standard area population

Population in the standard area

Population in the location of interestX

By matching age-sex

information

This work …• Estimate a time-series of ward populations using various methods

• Use outputs in SMRs

• Address denominator uncertainties

Research definitions• Small area: electoral wards (caveat)

• Mortality measure: indirect SMRs (caveat)

• Time period: annual mid-year estimates 1990-1998

• Geography: 1998 wards in GOR East

• Output detail: age-groups (11) and sex (2)

• Data acquisition: nationally consistent, public domain sources

• Base population: Estimating with Confidence Populations (EwCPOP) based on 1991 Census (caveat)

Steps to achieve this …Input data preparation

• Geographical harmonisation• Temporal harmonisation• Single year of age

Estimation methods• Indicator of sub-district, ward level change (electorate)• Cohort-component

Optional enhancements• Allowances for special sub-populations• Hybrid methods• Constraints

Standardised mortality ratios• Use ward age-sex estimates as populations at risk

2001 Census implications?

Geographical harmonisation

Postcode locations as building-bricks: assumptions• Residential postcode distribution is a proxy for population distribution (enhanced by household or address counts)• At a point in time a set of postcodes constitutes a ward

Haldens 1991 Haldens & Panshanger 1998

Temporal harmonisation

J F M A M J J A S O N D

Population estimates needed for the mid-year

ONS mid-year estimates

Electorate

Census

Vital statistics

Disaggregation to single year of age• For annual ageing-on• For aggregation into appropriate age-groups

Estimation methods data scheme

Data at time t

Data at time t + 1

Males Females

Age 0-90+ Age 0-90+

Wards (within LA

district)

? ?

LA district totals

Ward totals

Electorates as sub-district indicator

147500

150000

152500

155000

157500

160000

1990 1991 1992 1993 1994 1995 1996 1997 1998

Dis

tric

t p

op

ula

tio

ns

3000

6000

9000

Ele

cto

rs

District Electorate 1 Electorate 2

Electorates as sub-district indicator• Annual time-series available, but• Collected 10th October• Only adult ages• Variable enumeration space & time

Indicators of change

ONS MYEs• Annual mid-year time-series available• Age-sex detail, but• Only district level

Apportionment, additive & ratio methods

Data at time t

Data at time t + 1

Electorate derived ward totals

ONS district MYEs

Electorate derived ward totals

ONS district MYEs Change betweent & t + 1

Apply previous age structure &/or constrain

to MYE

Cohort-component method (includes Vital Statistics)

Data at time t

Data at time t + 1

+ - + -

Ageing-onBirths Deaths In-migration Out-migration

ONS district MYEs

Electorate derived ward totals

Cohort-component enhancement: Suppressed aging-on of special populations• Students

• Armed forces

• Communal establishments

Method option: Constraints

Ward age-sex estimates are controlled to sum to district-level age-sex information, ONS annual MYE

• Larger area estimates tend to be more reliable

• Ensures consistency with ONS published data & thus …

• More acceptable, but …

• Some LAs disagree with the ONS MYE

Wards in LA

district

FemalesMales1

n

Age-group (column) totals

Ages

0 90+ 0 90+

Age-group district-level constraints

Ward (row) totals

Ward constraints

Constraints and Iterative Proportional Fitting (IPF)

t + 1 initial age-sex estimates

Estimation methods

MYE Apportion-ment

Additive Ratio IPF Cohort-component

Without Vital Statistics &ageing on

District District District District District/ward

-

- Unconstrained Unconstrained Unconstrained - -

With Vital Statistics &ageing on

- - - District District/ward

District, ward, IPF

- - - Unconstrained - Unconstrained

With Vital Statistics,ageing &special populations

- - - District District/ward

District, ward, IPF

- - - Unconstrained - Unconstrained

Estimation methods & options

Strategy Method / option Population at risk

Do nothing approach

Use the ward populations from EwCPOP for 1991 in all subsequent years

EwCPOP

Minimal approachEwCPOP 1991 constrained to ONS MYEs for each year

ONS-MYE

Simpler methodRatio method with initial age-sex counts constrained to be consistent with ONS MYEs for each year

Ratio-constrained

More complex methods

Cohort-component including births, deaths and ageing and hybrid with IPF

CC-IPF

Cohort-component with gross migration flows and allowances for special populations and hybrid with IPF

CC-mig-sp-IPF

Many method / option combinations …Strategy for the choice of population at risk

Differences in estimate outputs …

Differences in outputs (1991 cf 1998)

Newnham: simpler methods constrained Coggeshall: simpler methods constrained

Coggeshall: cohort-component, plus migration and special populations

Newnham: cohort-component, plus migration and special populations

Differences in outputs (1991 cf 1998)

abs(1991 - 1998)

1991

Most variation in estimate outputs for:

• Youngest ages

• Young adults

• Most elderly

* 100

Using 1998 outputs in SMR calculations (Newnham)

Using 1998 outputs in SMR calculations (Newnham)

Smaller base population leads to lower expected

Student ages suppressed, elderly enhanced

Similar structure to base, total & elderly enhanced

Structure erroneously aged-on

Students enhanced, elderly suppressed

Smaller base population leads to lower expected

Student ages suppressed, elderly enhanced

Similar structure to base, total & elderly enhanced

Structure erroneously aged-on

Students enhanced, elderly suppressed

Lower expected leads to higher SMR

Higher expected leads to lower SMR

Youthful population leads to lower expected & higher SMR

Using 1998 outputs in SMR calculations (Newnham)

Comparison of 1998 SMRs: cf no population change

Are the differences enough to make a difference?!?Overlapping SMR confidence intervals?

• Yes, but observations small numbers leading to wide CIs

Do wards fall in the same SMR quintile?

Ranking by SMR:• Quintile 1: 29% wards consistently most healthy • Quintile 5: 6% wards least healthy

Differently estimated populations at risk and SMRs …• If a larger population is estimated by a method compared with another, but with the same age-sex structure, a lower SMR results because more events are expected (and vice versa)

• If a method estimates an older population structure than another, a higher expected is calculated, resulting in lower SMRs (and vice versa)

• Population size is more critical in simpler methods (as little or no new age information)

• Poorly specified cohort-component models tend to result in lower SMRs, because incorrectly aged-on populations lead to higher expected mortality

• Fully specified cohort-component models tend to result in greater range of SMRs, due to populations kept youthful in certain locations by migration data and suppressed ageing of sub-groups (proxy for migration)

• Areas with the best health consistently have lowest SMRs calculated

• Areas with the very worst health similarly identified but not the same consistency

• Fair level of tolerance in SMRs for all-ages

• Not necessarily the case with age-specific mortality rates (Rees et al., 2003a)

Following 2001 Census outputs (& rebased MYEs) …• Uncertainty in the EwCPOP base population used

• Uncertainty in the annual district level ONS MYEs used as constraints

In the light of the 2001 Census outputs …• Uncertainty in the annual national level ONS MYEs used for ASMRs

National ASMRs differ

Populations at risk differ

Thus: Expected changes

Events don’t change

SMRs alter

Uncertainty in SMR calculations …

SMR = 100 x Deaths in a location of interest

Mortality data for the ward (VS4)

Mortality data at national level

Population estimate at national level

Population estimate for the ward

Deaths in a standard area population

Population in the standard area

Population in the location of interestX

Uncertainty in estimated populations at risk• By total size & by age

Newnham

• Maximum

• Average

• Minimum

• CC-mig-sp-IPF

Coggeshall

• Maximum

• Average

• Minimum

• CC-mig-sp-IPF

No consideration here for rebasing MYEs!

Uncertainty in SMR calculations …

How confident can we be in our SMR results?

Confidence limits (c. 95%) are calculated using:

The assumption is that the ‘expected’ is reliable

But it is not!

Event counts may well be more reliable!!

Expected

ObservedSMR 10096.1 (or Byar’s

approximation)