1

Measuring & Managing Health Services: The Balanced Scorecard

David Peters, Director, Future Health David Peters, Director, Future Health Systems Research Consortium Systems Research Consortium

Sept 28, 2006Sept 28, 2006

2

“What gets measured gets managed. What gets managed gets done.”

Tom PetersTom Peters

3

Would This Measurement Describe Your Child?

4

Why Have a Balanced Scorecard?

Health sector is complex, has many components

Need an efficient way to assess multiple objectives

Overloaded with different types of reports

Stakeholders demand vigilance Poor measurement can lead to crisis

5

Balanced Scorecard – Original Definition

“A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organization’s strategy.”

Kaplan & Norton, 1996

6

Scorecard Domains

Which dimensions should we Which dimensions should we measure?measure?

7

Balanced Scorecard: General Domains

Customer Results Internal Processes Staff and Organizational Growth Financial Results Vision & Strategy

8

Qualities of Good Scorecard Measures

Valid Reliable Balanced Easily Understood Intermediable Agreed Upon Limited

Specific Measurable Achievable

Targets Relevant Timely

9

Presenting Scorecards: Performance Dashboards

Simple to see & interpret Multi-dimensional measures of

critical indicators of overall performance

Allows for the early detection of problems and successes

Can be benchmarked against other data

10

Afghanistan National Health Services Performance Assessment (NHSPA)

11

Developing the Balanced Scorecard

Frontline providers, NGOs, MOPH, donors toagree on: Purpose of BSC Domains to measure Unit of analysis Process & frequency of review/decisions Principles for benchmarking Short-listing indicators based on face

validity, importance, reliability

12

Afghanistan Balanced Scorecard Domains

1. Patient & Community Results2. Staff Results3. Capacity for Service Provision4. Service Provision Results5. Financial System Results6. Overall Vision Results

13

Measuring perceptions

Many survey questions sought perceptions. But how to ask?

Likert scale couldn’t be understood

Money?

ExcellentGoodFairPoor

The Naanogram

14

 2004 NHSPA

2005 NHSPA

Provinces 33 30

Facilities 617 628

Basic Health Centers 323 368

Comprehensive HCs 243 217

District Hospitals 51 43

Observations 5719 5863

Exit Interviews 5597 5863

Health worker Interviews 1553 1458

CHW Interviews 167 306

Afghanistan Sample for Balanced Scorecard

Afghanistan NHSPA Provincial Balanced Scorecard Measure

National Median

Lower Benchmark

Upper Benchmark Badakhshan Badghis Baghlan Balkh Bamyan Farah

A. Patients & Community

1 Overall Patient Satisfaction % 83.1 66.4 90.9 86.4 76.9 90.9 84.7 97.9 82.82 Patient Perception of Quality Index % 76.0 66.2 83.9 77.6 66.2 82.2 80.0 84.4 69.43 Written Shura-e-sehie activities in community % 34.2 18.1 66.5 35.6 0.0 34.2 17.7 34.5 73.2

B. Staff

4 Health Worker Satisfaction Index % 63.5 56.1 67.9 63.5 57.6 67.9 68.3 61.4 54.45 Salary payments current % 76.7 52.4 92.0 54.9 91.8 45.8 53.3 91.4 97.7

C. Capacity for Service Provision

6 Equipment Functionality Index* % 65.7 61.3 90.0 69.6 62.2 57.5 67.3 75.8 66.37 Drug Availability Index % 71.1 53.3 81.8 52.9 50.1 72.8 56.1 85.6 9.88 Family Planning Availability Index % 61.4 43.4 80.3 54.2 57.9 70.4 64.9 82.7 0.09 Laboratory Functionality Index (Hospitals & CHCs) % 18.3 5.6 31.7 31.7 3.8 15.2 0.0 37.0 0.0

10 Staffing Index -- Meeting minimum staff guidelines % 39.3 10.1 54.0 38.0 22.4 42.7 45.8 53.0 57.111 Provider Knowledge Score % 53.5 44.8 62.3 48.6 41.6 49.3 54.0 69.0 45.512 Staff received training in last year % 39.0 30.1 56.3 68.9 50.9 39.0 52.4 35.5 37.213 HMIS Use Index % 67.7 49.6 80.7 60.9 62.7 40.0 72.9 67.7 72.414 Clinical Guidelines Index % 34.8 22.5 51.0 18.3 25.5 29.9 16.4 41.9 59.515 Infrastructure Index % 55.0 49.3 63.2 63.2 49.7 50.0 58.3 57.9 76.716 Patient Record Index % 65.6 56.1 92.5 51.5 98.5 80.7 97.3 64.5 97.117 Facilities having TB register % 15.8 8.3 26.6 32.5 27.0 16.1 16.4 0.0 4.3

D. Service Provision

18 Patient History and Physical Exam Index % 70.6 55.1 83.5 54.2 71.7 55.1 85.4 83.6 52.019 Patient Counseling Index % 29.6 23.3 48.9 23.3 40.4 29.3 55.3 33.2 16.020 Proper sharps disposal % 62.2 34.1 85.0 64.4 34.1 76.9 75.1 85.0 67.821 Average new outpatient visits per month (BHC>750 visits) % 22.2 6.7 57.1 27.3 10.0 27.3 71.4 22.2 0.022 Time spent with patient (> 9 minutes) % 18.0 3.5 31.2 21.0 30.7 1.2 27.3 12.8 18.023 BPHS facilities providing antenatal care % 62.0 28.9 82.8 28.9 49.4 49.7 67.2 88.1 82.824 Delivery care according to BPHS % 25.4 10.5 39.3 38.0 36.2 10.5 39.3 38.0 57.1

E. Financial Systems

25 Facilities with user fee guidelines % 90.6 80.3 100.0 94.8 95.6 95.9 28.9 86.1 100.026 Facilities with exemptions for poor patients % 84.7 64.4 100.0 68.5 54.6 69.3 84.3 95.6 93.9

F. Overall Vision

27 Females as % of new outpatients % 55.2 46.5 59.7 46.9 45.9 56.0 55.1 55.2 59.028 Outpatient visit concentration index CI (-1 to 1) -0.010 0.041 -0.055 0.021 0.024 -0.038 0.025 -0.076 -0.03629 Patient satisfaction concentration index CI (-1 to 1) 0.002 0.020 -0.018 -0.019 0.000 0.003 -0.007 -0.005 0.020

Composite Scores

30 Upper Benchmarks Achieved % 17.2 10.3 30.8 17.2 6.9 6.9 20.7 34.5 31.031 Lower Benchmarks Achieved % 82.8 75.9 89.7 86.2 79.3 86.2 86.2 96.6 72.4

16

Mean Score(29 Provinces)

2004 2005 Difference

1. Overall patient satisfaction 81.3 84.1 2.8

2. Patient perceptions of quality

74.1 76.5 2.4

3. Written Shura-e-Sehie Activities 39.9 52.4 12.5

Patients & Community Perspectives

17

Mean Score(29 Provinces)

2004 2005 Difference

4. Health worker satisfaction 61.9 64.0 2.1

5. Salary payments current 68.9 83.9 15.0

Staff Perspectives

20

Mean Score(29 Provinces)

2004 2005 Difference

18. Patient history and physical exam

69.5 74.7 5.1

19. Patient counseling

36.7 38.1 1.4

20. Proper sharps disposal

58.5 48.9 -9.6

21. New outpatient visits (>750/month)

27.6 42.8 15.2

Service Provision

23

Mean Score(29 Provinces)

2004 2005 Difference

27. Females as % of new outpatients 53.7 56.3 2.6

28. Outpatient visit concentration index

-0.008 -0.021 -0.013

29. Pt. satisfaction concentration index

-0.001 0.004 0.005

Overall Vision

24

Identifying Exceptional Provincial Performance

Top Performers

 Change 2004-05

Wardak 18.0

Balkh 16.0

Ghor 15.5

Faryab 12.0

Saripul 11.1

Bottom Performers

 Change 2004-05

Herat -2.2

Paktika -3.0

Nangarhar -4.7

Nimroz -5.5

Nuristan -13.7

25

Balanced Scorecard: Improvements Progress has been made in most areas

Improvements in 22 out of 29 indicators

Areas of largest improvement include: Shura-e-Sehie activity Salary payments current Laboratory functionality Drug availability Meeting minimum staffing guidelines

26

Balanced Scorecard: Problems Improvement not consistent across all

provinces or aspects of service delivery

Areas of concern:Time spent with patientsFacility infrastructure Sharps disposal User fee guidelines Patient records

27

Balanced Scorecard on Contract Performance: Cumulative Ratings

2004 2005 Difference

Performance Based Contracts (WB)

52.3 64.4 12.1

Contracts with Technical Assistance (USAID)

53.4 63.6 10.2

Ministry of Health Strengthening (WB)

53.7 61.8 8.1

Contracting Without Performance (EU)

54.2 57.7 3.5

Usual Management 51.5 48.5 -3.0

28

Some Final Thoughts

29

Successful Scorecards

Focus on significant, success-determining measures of organization’s total performance

Based on timely information Predictive (leading) indicators more

useful than trailing indicators

30

“You can’t fatten a cow by weighing it”