Evaluation of the

Mother and Infant Health Project

Olena Nizalova

University of Kent

Maria Vyshnya

Emory University

* This study had been funded by the Global Development Network and Bill & Melinda Gates Foundation, within the GDNs Fifth Global Research Project “Promoting Health Innovative Programs from the Developing World: Towards Realizing the Health MDG in Africa and Asia”

Kyiv, 30 June 2016

Evaluation of the

Mother and Infant Health Project

Full discussion paper can be found at http://repec.kse.org.ua/pdf/KSE_dp18.pdf

Published version in Health Economics

http://hdl.handle.net/10.1002/hec.1609

Twitter: @olniz

Facebook: https://www.facebook.com/olena.nizalova

3

Outline

• Motivation

• Impact evaluation:– Methodology

– Data

• Results:– Estimated Results

– Pre-treatment test (time dimension)

– Placebo Outcomes

– Metropolitan Areas

• Cost-Benefit Considerations

• Conclusions

4

Motivation

• Quality vs. availability of care (Barber and Gertler 2002, Sodenmann et al. 1997)

• Problems with the identification of quality

impact:

– Separating quality from availability

– Measurement of outcomes

– Outcome base magnitude

– Quality measures reported by patients

5

MIHP in Ukraine

• High coverage of basic prenatal and obstetric care (quality vs. quantity effect)

• Administrative medical statistics (compulsory, uniform, publicly available)

• Relatively high maternal and infant mortality rate / poor health (significant effect of the program)

• Quality improvement is measured as MIHP participation and is unrelated to the perceptions of the patients.

6

MIHP in Ukraine

(USAID funded)

Personnel Trainings at Local Maternities

(2002-2006) – 13 sites in 8 regions

• Perinatal and postpartum care - new WHO practices (encouragement of partner/family member presence in the delivery room, “warm chain practices”, free position during delivery, avoidance of C-sections, amniotomies, and episiotomies, etc.)

• Post-partum and post-abortion counseling onfamily planning methods

• Requirement to provide trainings for the medical practitioners from the same region

7

Regression Design

Hrt = β0 + βPPrt + TtβT + RrβR +

+ O*TtβT + βXXrt + urt

Prt – treatment (MIHP rayon , MIHP oblast, MIHP time

dummies, MIHP % coverage)

Tt , Rr ,,O – time, rayon, and oblast dummies

Xrt – other programs

DD (DDD with placebo outcome)

8

Data

Regional subsidiaries of

• The Center of Medical Statistics of Ukraine

(CMSU)

Annual rayon-level data for 2000-2006

Urban Rayons:

• 244 urban rayons (13 - treatment)

• Observations - 1612 (6.6 observation per

rayon on average)

9

Infant MortalityMIHP Other MIHP MIHP Other

rayon Programs rayon oblast Programs

(1) (2) (3) (4) (5)

Infant Mortality -3.13*** -2.12 -2.99 0.15 -2.12

(0.90) (1.55) (2.48) (2.55) (1.56)

Stillbirths -1.58*** -0.20 -1.01 0.59 -0.18

(0.54) (0.67) (1.61) (1.61) (0.67)

Perinatal -2.37*** -2.37** -0.88 1.56 -2.33**

(0.72) (1.01) (1.41) (1.44) (1.01)

Early Neonatal -0.77 -2.05*** 0.13 0.95 -2.02***

(0.48) (0.77) (0.98) (0.94) (0.77)

Late Neonatal -0.54 -1.11** -0.60 -0.06 -1.11**

(0.35) (0.47) (0.78) (0.80) (0.47)

Postneonatal -0.76 0.21 -1.94 -1.24 0.17

(0.62) (0.86) (1.55) (1.61) (0.87)

10

Infant Mortality By CauseMIHP Other MIHP MIHP Other

rayon Programs rayon oblast Programs

(1) (2) (3) (4) (5)

Infection -1.43 3.08 4.93 6.64 3.26

(1.97) (2.48) (4.76) (4.96) (2.50)

Nervous -1.85 3.94* 4.96 7.11 4.13*

(1.29) (2.20) (6.52) (6.70) (2.20)

Respiratory 0.39 -0.91 -9.31* -10.12** -1.17

(1.95) (2.39) (5.11) (5.13) (2.40)

Congenital -8.12* -0.46 -21.15 -13.60 -0.81

Anomalies (4.27) (6.12) (19.89) (20.05) (6.11)

Perinatal -14.10*** -9.62 -20.64** -6.83 -9.80

Deviations (5.22) (8.63) (9.53) (9.49) (8.65)

11

Infant MorbidityMIHP Other MIHP MIHP Other

rayon Programs rayon oblast Programs

(1) (2) (3) (4) (5)

Infant Morbidity -1.71 -0.46 -35.57 -35.36 -1.37

(8.57) (11.74) (43.54) (43.47) (11.77)

Infection -0.25 0.59 1.97 2.32* 0.65

(0.60) (0.64) (1.52) (1.32) (0.64)

Nervous 0.00 0.74 -0.09 -0.10 0.74

(1.19) (2.18) (2.91) (2.70) (2.19)

Respiratory 6.71 -17.08 -42.86 -51.77 -18.42

(7.02) (12.22) (44.47) (44.42) (12.32)

Congenital -0.03 -0.10 -0.10 -0.07 -0.10

Anomalies (0.04) (0.11) (0.08) (0.08) (0.11)

Perinatal -0.53** 0.53 0.06 0.61 0.55

Deviations (0.21) (0.43) (0.45) (0.45) (0.43)

12

Maternal Mortality, Intermediate Outcomes

MIHP Other MIHP MIHP Other

rayon Programs rayon oblast Programs

(1) (2) (3) (4) (5)

Maternal

Mortality

-7.10 -0.60 -63.06* -58.45* -2.11

(9.24) (16.95) (34.02) (33.73) (17.08)

Normal 10.12*** 0.04 18.79*** 9.06** 0.28

Deliveries (2.06) (3.73) (4.79) (4.15) (3.69)

C-sections -2.48*** 0.75 -4.85*** -2.48 0.69

(0.70) (1.07) (1.75) (1.56) (1.05)

Early Neonatal 2.09** -0.11 2.89* 0.83 -0.08

Visits (0.85) (1.08) (1.58) (1.54) (1.08)

Late Toxicosis -1.92** -1.73 -2.87** -1.00 -1.76

(0.80) (1.37) (1.19) (1.10) (1.37)

13

Complicated Deliveries

MIHP Other MIHP MIHP Other

rayon Programs rayon oblast Programs

(1) (2) (3) (4) (5)

Urinary-Genital -1.82 -0.44 -4.38** -2.67 -0.51

(2.01) (1.41) (2.02) (1.95) (1.44)

Anemia -5.02*** -5.05** -11.27*** -6.53* -5.22**

(1.86) (2.54) (3.69) (3.62) (2.54)

Blood Circulation -1.39** -0.45 -2.19* -0.84 -0.47

(0.57) (0.72) (1.17) (1.04) (0.71)

Veins -0.54** -0.26 -0.60 -0.06 -0.26

(0.24) (0.33) (0.76) (0.75) (0.33)

Thyroid Gland -0.96 -0.60 -4.34 -3.53 -0.69

(1.41) (1.66) (4.84) (4.80) (1.66)

14

Maternal Health: Time Dimension

Before After

Outcomes 3 years 2 years 1 year 1st year 2nd year 3d year

Maternal -16.52 9.23 21.54 4.25 -17.12 -4.78

Mortality (18.22) (14.37) (14.43) (11.91) (14.72) (16.79)

Normal -0.62 -1.73 0.63 10.03*** 12.30*** 4.23

Deliveries (2.61) (2.38) (2.85) (3.40) (3.23) (3.08)

C-sections 0.76 0.91 -0.48 -1.94** -2.68** -2.06

(1.06) (0.85) (0.76) (0.97) (1.20) (1.74)

Early -0.90 -0.22 -0.60 2.70** 1.22 -0.34

Visits (0.88) (1.09) (0.93) (1.24) (1.25) (1.51)

Late -1.23 -1.51 -1.80 -2.58** -3.19** -3.68

Toxicosis (1.21) (1.30) (1.11) (1.20) (1.56) (2.35)

15

Placebo Outcomes

MIHP Other_prog

DD DDD DD DDD

(1) (2) (3) (4)

Diabetis Morbidity 3.56 0.35 -12.81 -11.65

(7.02) (1.51) (14.41) (18.55)

Hepatitis 7.55 -3.69 -13.99 -10.36

(17.58) (3.30) (14.73) (14.05)

16

Patient Mix Change?

Metropolitan Rayons:

• 93 urban rayons (11 - treatment)

• Observations - 602 (6.4 observation per

rayon on average)

17

Infant Health: Metropolitan

All Urban Metropolitan

MIHP MIHP Other MIHP MIHP Other

rayon oblast Programs rayon oblast Programs

Infection 4.93 6.64 3.26 -2.35 -1.55 0.43

(4.76) (4.96) (2.50) (3.95) (4.85) (2.82)

Nervous 4.96 7.11 4.13* -1.51* -1.62 0.56

(6.52) (6.70) (2.20) (0.77) (1.81) (2.94)

Respiratory -9.31* -10.12** -1.17 2.35 3.93 -0.06

(5.11) (5.13) (2.40) (2.80) (3.75) (2.31)

Congenital -21.15 -13.60 -0.81 -54.51*** -58.86*** -6.85

Anomalies (19.89) (20.05) (6.11) (8.12) (9.95) (5.92)

Perinatal -20.64** -6.83 -9.80 -21.22** -12.86 -16.01

Deviations (9.53) (9.49) (8.65) (9.22) (11.13) (9.89)

18

Maternal Health: Metropolitan

All Urban Metropolitan

MIHP MIHP Other MIHP MIHP Other

rayon oblast Programs rayon oblast Programs

M Mortality -63.06* -58.45* -2.11 -21.58* -19.87 -1.83

(34.02) (33.73) (17.08) (11.53) (16.18) (15.70)

N Deliveries 18.79*** 9.06** 0.28 37.34*** 31.37*** 1.36

(4.79) (4.15) (3.69) (2.90) (3.62) (3.78)

C-sections -4.85*** -2.48 0.69 -11.22*** -9.71*** -0.20

(1.75) (1.56) (1.05) (2.64) (2.72) (1.12)

Early 2.89* 0.83 -0.08 4.16*** 0.66 -1.12

N Visits (1.58) (1.54) (1.08) (1.33) (1.92) (1.41)

Late -2.87** -1.00 -1.76 -5.76** -4.72* -3.10*

Toxicosis (1.19) (1.10) (1.37) (2.45) (2.76) (1.73)

19

Cost-Benefit Considerations: Summary

• Costs per maternity per year:

- $92,308 (MIHP Project costs, 2002-2006 Phase: $6mln, 13 sites)

- ??? (Maternity building renovation, equipment)

• Benefits per maternity per year:

+ ??? (2 mothers saved)

+ ??? (10 – 3 respiratory and 7 perinatal deviations –newborns saved)

+ $4,646.60 (CS↓)

+ $117,374.06 (Saved cost due to change in practices)

20

Conclusions

1. Drastic improvements in maternal health:Decrease in :

• Maternal mortality

• C-sections

• Share of deliveries and post-partum complications (maladies of blood circulation system, veins, thyroid gland

diseases, anemia, urinary-genital maladies, late toxicosis)

Increases in:

• Share of normal deliveries

• Share of women attended doctors prior to 12 weeks of gestation

21

Conclusions

2. Improvements in infant health:Decrease in:

• Infant mortality

(Stillbirths, Perinatal mortality, Mortality resulted from congenital anomalies and perinatal deviations)

• Infant morbidity resulted from perinatal deviations

3. Importance of trainings:• Improvements in maternal and infant health in MIHP

oblast

• The “oblast” effect is lower compared to “MIHP rayon” effect

22

Conclusions

4. Seem to be quite Cost-effective (CBR

1:97), but thorough cost-benefit analysis is

required.

23

Further Research

• Census of maternities to collect

information on major equipment

purchases, renovations, and patient

referrals

• Female exit survey to identify whether and

what kinds of MIHP promoted practices

are used in a particular maternity (both in

treated and control rayons)

24

MIHP Ukraine: 2006-2010 phase

25

Program Importance

Indicator UkraineEast Asia/

South Asia

North Africa/

Sub-Saharan

Africa

EU

Maternal

Mortality

(2000)35 116/ 567 162/ 916 9

Infant Mortality

(2003)15 32/ 66 43/ 101 4

Abortion ratio

(2000)53 26.8 5 20

26

Impact Evaluation: Data

Regional subsidiaries of

• The Center of Medical Statistics of Ukraine (CMSU)

Annual rayon-level data for 2000-2006

Urban Rayons:

• 244 urban rayons (13 - treatment)

• Observations - 1612 (6.6 observation per rayon on average)

Metropolitan Rayons:

• 93 urban rayons (11 - treatment)

• Observations - 602 (6.4 observation per rayon on average)