estimating return on investment for public health improvements

Estimating Return on Investment for Public Health Improvements Tutorial on Using the new Tool

Karl Ensign, Director of Evaluation

Association of State and Territorial Health Officials (ASTHO)

Beta Testers:

Brynn Riley, Maine

Josh Czarda, Virginia

Susan Logan, Connecticut

Tool Developed by Dr. Glen P. Mays, University of Kentucky

June 2013

Helps answer the following

questions:

Are we making the right

investments?

Are we becoming more efficient?

What bang are we getting for our

buck?

What is our budget

accomplishing?

Are we being good stewards?

Must be done thoughtfully

and carefully

ROI is one way of

measuring and

communicating public

health effectiveness in a

manner that is particularly

salient for policymakers,

funders, administrators and

the general public

ROI is a Specialized Form of Analysis

Outcomes are monetized:

Compares the costs of an intervention with its benefits – in

financial terms

Yields the net return on investment – over time

Sensitivity analysis can be conducted – different levels of

investments and their benefits

ROI Calculated:

Net benefit = Benefits – Costs

ROI = Benefits – Costs

Costs

Hypothetical values:

• $5 = ($400 + $500 + $300) – ($150 + $50)

($150 + $50)

Or …

“…a dollar spent on pediatric immunization is estimated to save $5 in treating preventable

illness.”

WHERE AND HOW HAS ROI BEEN USED IN PUBLIC HEALTH?

Public health programs (injury prevention, tobacco prevention and control)

Aggregate public health spending (Glen P. Mays, UKY)

Improvement projects undertaken by agencies (primary ASTHO focus)

• New data probably need to

be collected

• An investment may take

years to produce benefits

• Benefits may accrue outside

the agency (population

health)

• Benefits may be difficult to

link back to a specific public

health program(s) or

function(s)

Challenges from an Agency Perspective

ROI of QI – Advantages and Disadvantages

ROI is more immediate

Timeframe is shorter

ROI accrues more directly

to the agency

Tends to be narrower in scope

Return is more modest

Funding from Prevention & Public Health Fund, ACA

Measures ROI for improvement projects

QI projects undertaken through the National Public Health

Improvement Initiative (NPHII)

ASTHO led workgroup provided input and guidance

CDC, state and local agencies, foundations, academia

Developed by Glen Mays, University of Kentucky

Beta tested by Connecticut, Maine, and Virginia

Pathways to Realizing ROI for

QI

Reductions in standard operating

costs

• Greater efficiencies realized

Revenue enhancements

• Increased cost reimbursement

Increased productivity of agency

functions

• Increased service encounters

Decreased time to produce outputs

• Reduced cycle time process

Achievement of health outcomes

• Tool accommodates this

Cost Category Pre-

Implementation(

Baseline)

Post-

Implementation

(Year 1)

Year 2

…

Personnel Costs

Non-Personal

• Contracted Services

• Office Operations

• Facilities/Maint/Rent

• Communications

• Equipment

• Construction/Renovation

Other Direct Costs

Indirect Costs

Planning and Development Costs

Cost Category Pre-

Implementation(

Baseline)

Post-

Implementation

(Year 1)

Year 2

…

Personnel Costs

Non-Personal

• Contracted Services

• Office Operations

• Facilities/Maint/Rent

• Communications

• Equipment

• Construction/Renovation

Other Direct Costs

Indirect Costs

Routine Operating Costs

Outcomes/Outputs

Cost Category Pre-

Implementation

Post-

Implementation

(Year 1)

Year 2

…

Service Units Delivered

Required Production Time

Target Population Reach

Other Outcomes/Outputs

How the Tool Calculates ROI

ROI = Benefits – Costs

Costs Improvements in Routine Operations – Investment Costs

Investment Costs

(Routine Operations + Other Outcomes) – Investment Costs

Investment Costs

Incorporates Standard Accounting Practices in ROI Calculation

Amortization

The cost of an investment should not be

absorbed entirely in the first year

Amortization rate spreads the agency’s

cost/investment over the useful life of the

product

Present value

The relative worth of a single dollar changes over time

Accurate comparisons are made by applying a discount rate

(inflation) to

• Costs

• Returns

Tool Can Be Used throughout Project

Prospectively – Planning Phase

Implementation Phase

Retrospectively – Post Implementation

WHEN & HOW TO USE THE TOOL

Ask ourselves the Right Questions

Is ROI the corrective evaluative tool? Have outcomes been defined and agreed upon?

Have performance measures been set and tracked?

Has effectiveness been evaluated?

How will the ROI results be used? Internally

• Allocation

• Investment maximization

Externally

• Budget justification

Apply ROI Correctly

Build evaluation methods – including ROI – into

project inception

Clearly specify intended purpose and use of ROI

Think about how results will be communicated

sooner rather than later

Conduct ROI through a

transparent process

Conduct ROI through

an inclusive process

Implementers and end

users of the analysis

LET’S SEE HOW IT WORKS!

Testing the New ROI Tool in Maine: A Prospective

Look

Brynn Riley, MS

Maine Center for Disease Control and Prevention

June 7, 2013

6/19/2013 22

Maine

6/19/2013 23

• 31,000 square miles

• 1.3 million people

• Average: 43 people per square mile (ppsm). o US = 87 ppsm.

• 16 counties which comprise of 8 Districts.

• Each District has a District Liaison o Role: to increase collaboration for a greater consistency and equity in the

statewide delivery of public health.

o They serve as liaisons between state planning and public, non-profit and

business sectors.

Background on QI Initiative • AIM: The District Liaisons will reduce their overall

travel costs to and from meetings by 50% by March

1, 2014.

• Plan o Identify the causes of high travel costs

o Identify alternative solutions to travel that address the highest cost areas

o Apply the ROI tool to analyze the business case

o Put a plan in place that provides a sustainable approach

o Track the new travel costs

o Evaluate the effectiveness of the new approach

6/19/2013 24

Baseline Data

All Other

District

Liaison

RT mileage to

Augusta Mileage cost

Lodging &

per diem

Total cost no

overnight

Total cost w/

overnight

% of time

traveling

DL 1 500 233.00$ 141.00$ 233.00$ 374.00$ 60

DL 2 330 158.20$ 128.00$ 158.20$ 286.20$ 7

DL 3 400 176.00$ 128.00$ 176.00$ 304.00$ 25

DL 4 400 176.00$ 128.00$ 176.00$ 304.00$ 25

DL 5 150 66.00$ 66.00$ 66.00$ 10

DL 6 178 83.32$ 83.32$ 83.32$ 10

DL 7 80 37.70$ 37.70$ 37.70$ 15

DL 8 62 29.78$ 29.78$ 29.78$ 1

2100 960.00$ 525.00$

1-day with overnight stays

If all travel: 960.00$ 1,485.00$

Average salary with fringe: $90,000

Cost of Quarterly Trip to Augusta for DL meeting

DL travel costs

Walk-through of ROI Tool

and Definitions

Time Periods:

Pre-Implementation

(Plan), Implementation (Do and Check),

Post-implementation

(Act)

Gather Financial

Data:

Salaries, Supply costs,

Travel, etc.

Calculate Costs of Planning

and Intervention

Calculate Costs of

Operations

Calculate Costs/

Savings for Outcomes

ROI is Calculated

Using the ROI Tool

Project Investment Costs

27

From To From To

Cost Category Reporting Period: 1/1/2012 12/31/2012 1/1/2013 2/28/2013

Personnel (see below) List by name or by category FTE % Cost FTE % Cost

District Liaison Cumberland 12.00% 37,500.00$ 4,500.00$ 0.00% -$

District Liaison Downeast 5.00% 37,500.00 1,875.00 0.00% -

0.00% - 0.00% -

Office Manager 1 OHE Division 0.00% - 10.00% 4,166.00$ 416.60

Office Manager 2 PHS Division 0.00% - 10.00% 4,166.00$ 416.60

Office Manager 3 Chronic Disease Division 0.00% - 10.00% 4,166.00$ 416.60

Office Manager 4 Infectious Disease Division 0.00% - 10.00% 4,166.00$ 416.60

Office Manager 5 Environmental Health Division 0.00% - 10.00% 4,166.00$ 416.60

0.00% - 0.00% -

0.00% - 0.00% -

Office operations

T ravel/conferencing Parking Passes ($3 pp per day) 18.00

Equipment

Software Temp WebEx License for testing $50

Software Web Ex License - annual 550.00

Subtotal direct costs 6,443.00$ 2,633.00$

Indirect cost percentage (see below)

Indirect costs (see below) -$ -$

Notes:

Total costs of investment in strategy 6,443.00$ 2,633.00$

Years of expected utility from pre-implementation investments (see below) 1.00

Amortized Pre-Implementation Investment Costs $6,636.29 $6,636.29

Pre-Implementation - Baseline Implementation Period 1

Salary and

Fringe

Salary and

Fringe

Staff time devoted to the project

Web EX training time

Represents 5 months of pre-implementaion work spread out over 12 mo's

Represents 8 weeks of implementation 3/1-4/15

Routine Operating Costs

From To From To From To

Cost Category Reporting Period: 1/1/2012 12/31/2012 1/1/2013 12/31/2013 1/1/2014 12/31/2014

Personnel (see below) Name / Notes FTE % Cost FTE % Cost FTE % Cost

District Liaison York - travel time 10.00% 90,000.00$ 9,000.00$ 8.00% 15,000.00$ 1,200.00$ 5.00% 90,000.00$ 4,500.00$

District Liaison Cumberland tarvel time 7.00% 90,000.00$ 6,300.00 5.00% 15,000.00$ 750.00 3.50% 90,000.00$ 3,150.00

District Liaison Downeast travel time 25.00% 90,000.00$ 22,500.00 20.00% 15,000.00$ 3,000.00 12.00% 90,000.00$ 10,800.00

District Liaison Penquis travel time 25.00% 90,000.00$ 22,500.00 20.00% 15,000.00$ 3,000.00 12.00% 90,000.00$ 10,800.00

District Liaison Western travel time 10.00% 90,000.00$ 9,000.00 8.00% 15,000.00$ 1,200.00 5.00% 90,000.00$ 4,500.00

District Liaison Aroostook travel time 60.00% 90,000.00$ 54,000.00 50.00% 15,000.00$ 7,500.00 30.00% 90,000.00$ 27,000.00

District Liaison MidCoast travel time 15.00% 90,000.00$ 13,500.00 10.00% 15,000.00$ 1,500.00 7.00% 90,000.00$ 6,300.00

District Liaison Central travel time 1.00% 90,000.00 900.00 1.00% 15,000.00$ 150.00 1.00% 90,000.00$ 900.00

0.00% - 0.00% - 0.00% -

Office Manager 1 OHE Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00

Office Manager 2 PHS Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00

Office Manager 3 Chronic Disease Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00

Office Manager 4 Infectious Disease Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00

Office Manager 5 Environmental Health Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00

Office operations

Travel/Mileage $990.8 per DL staff trip to Augusta (990.8 x 12) 11,889.00 990.8 one per quarter (January) 990.80 990.8 x 4, one per quarter 3,963.20

T ravel/conferencingParking passes ($3 per pass x 12 mtgs x 8 DLs) 288.00

Equipment

Software Web Ex License 550.00 550.00

Subtotal direct costs, operating 149,877.00$ 19,840.80$ 84,963.20$

Indirect cost percentage (see below)

Indirect costs, operating (see below) -$ -$ -$

Total costs, operating 149,877.00$ 19,840.80$ 84,963.20$

Salary and

Fringe

Salary and

Fringe

Salary and

Fringe

Pre-Implementation - Baseline Implementation Period I Implementation Period 2

Represents a decreasing average time spread out over 12 mo's

Measuring Outputs

6/19/2013 29

Measures of Production Time

TRUE Description: output/event and time frameTotal

events

Average

time (days)

Total time

(days)

Total

events

Average time

(days)

Total time

(days)

Total

events

Average time

(days)

Total time

(days)

Building new collaborative

partnerships through training

sessions with key stakeholders

Overall average travel time from office to Augusta:

DL's monthly visit to Me CDC (note: total travel time for all

DL's is 36.2 hrs / 8 = 4.525 days of travel for one mtg in

Augusta) 12.00 4.525 54.30 4.00 4.525 18.10 4.00 4.525 18.10

Building new collaborative

partnerships through training

sessions with key stakeholders

Overall average travel time from office to local meetings:

Total DL's local meetings avg 80/mo, (80x12 = 960

mtgs); avg 30 miles R/T @ 60 mph, 30 minutes is

.06265 of an 8 hr day is 58 days. 960.00 0.630 604.80 480.00 6.300 3,024.00 480.00 6.300 3,024.00

Baseline Post Period - Year I Post Period - Year 2Check here to use these measures in ROI calcuations

The travel time saved is used to provide district partner trainings.

Post Period – Year 3 data is the same as year 2.

Saved staff time invested in other activities produces the following value

ROI Analysis

6/19/2013 30

I. Analysis EXCLUDING Output, Production Time and Outcome Factors

Discount Rate 3%

Pre-Implementation Total

Baseline 1 2 3 (All Periods)

Investment in Project

Amortized Pre-Implementation Costs $6,443.00

Ongoing Implementation Costs 2,633.00 - -$

Total Annual Implementation Costs 6,443.00 2,633.00 - -

x Present Value Factors (see below) 1.00 0.97 0.94 0.92

Total Discounted Annual Implementation Costs 6,443.00$ 2,556.31$ -$ -$ 8,999.31$

Financial Gain/Loss Attributable to Project

Estimated Changes in Operating Costs (130,036.20)$ (64,913.80)$ (64,913.80)$

Estimated Changes in Revenue -$ -$ -$

Total Net Financial Gain/Loss $130,036.20 $64,913.80 $64,913.80

x Present Value Factors (see below) 0.97 0.94 0.92

Total Discounted Annual Net Gain/Loss 126,248.74$ 61,187.48$ 59,405.32$ 246,841.54$

Return on Investment Summary

Undiscounted Annual Net Cash Flows (6,443)$ 127,403$ 64,914$ 64,914$ 250,788$

Cumulative ROI 14.03 20.83 27.43 27.43

Net Present Value 237,842$

Internal Rate of Return 1929%

Implementation Period

ROI Analysis II. Analysis INCLUDING Output and Outcome Measures (Production Time, Reach, and Health-Related Outcome Factors)

Total



Amortized Pre-Implementation Costs $6,636.29 $6,636.29 $6,636.29 $6,636.29

Ongoing Implementation Costs $0.00 $2,633.00 $0.00 $0.00

Total Annual Implementation Costs $6,636.29 $9,269.29 $6,636.29 $6,636.29

x Present Value Factors (see below) $1.00 $0.97 $0.94 $0.92

Total Discounted Annual Implementation Costs $6,636.29 $8,999.31 $6,255.34 $6,073.15 $27,964.09


Estimated Changes in Operating Costs (130,036.20)$ (64,913.80)$ (64,913.80)$

Estimated Changes in Revenue -$ -$ -$

Total Net Financial Gain/Loss 130,036.20$ 64,913.80$ 64,913.80$

x Present Value Factors (see below) 0.97$ 0.94$ 0.92$

Total Discounted Annual Net Gain/Loss 126,248.74$ 61,187.48$ 59,405.32$ 246,841.54$

Non-Financial Gains/Losses Attributable to Project

Estimated Changes in Output Calculation not enabled - - -

Estimated Change in Cost per Unit Output - - -

Monetary Value of Output Gain/Loss -$ -$ -$

x Present Value Factors (see below) -$ -$ -$

Total Discounted Value of Output Gain/Loss -$ -$ -$ -$

Cumulative ROI including Value of Output -$ -$ -$ -$

Estimated Production T ime Gain/Loss (2,383.00) (2,383.00) (2,383.00)

Monetary Value of T ime Gain/Loss (121,135.83)$ (532,534.31)$ (532,534.31)$

x Present Value Factors (see below) 0.97$ 0.94$ 0.92$

Total Discounted Value of T ime Gain/Loss (117,607.61)$ (501,964.66)$ (487,344.33)$ (1,106,916.59)$

Cumulative ROI including T ime Cost 0.55$ (19.74)$ (30.76)$ (30.76)$

Implementation Period

Highlights of using the ROI Tool

• Benefits: o Used prospectively to assess potential ROI and build a business case for

the project

o Will be used throughout the project to compare estimates against actual

project costs

o Could be used to compare two multiple strategies

o Seems daunting at first, but once data is collected, calculations are done

for you

• Cautions: o Some of the data can be sensitive (i.e. salaries)

o Some data can be difficult to obtain

6/19/2013 32

Josh Czarda, Performance Improvement Manager, Virginia Department of Health

Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background

(A Medicaid Family Planning Program)

34

Post Partum

Birth Control

Other – Family Planning

LHD

External Provider

Registration

WebVision

DSS

Individuals Providers Enrollment Process

Service

Billing

Service Provide

d

STD Testing

WIC Clinic

Demographic, Age, Income,

Insurance

Possible - Prompt for Plan First. Clerk

still must know parameters

Ask Questions to Determine Eligibility

If eligible – fill out application/ self populates some

fields

If Interested Individual Signs & Dates

LHD Submits

Printed App to DSS

Eligible

No More

Needed

Application Given to

Individual

Not Interested

Individual Follows Up

Eligibility

LHD Bills

Medicaid

Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background

(A Medicaid Family Planning Program)

35

Post Partum

Birth Control

Other – Family Planning

LHD

External Provider

Registration

WebVision

DSS

Individuals Providers Enrollment Process

Service

Billing

Service Provide

d

STD Testing

WIC Clinic

Demographic, Age, Income,

Insurance

Possible - Prompt for Plan First. Clerk

still must know parameters

Ask Questions to Determine Eligibility

If eligible – fill out application/ self populates some

fields

If Interested Individual Signs & Dates

LHD Submits

Printed App to DSS

Eligible

No More

Needed

Application Given to

Individual

Not Interested

Individual Follows Up

Eligibility

LHD Bills

Medicaid

No Incentive

Title X & Payment Issues

System Understanding

& Function

Tracking & Follow Up Capability

Application Process/

Form Length

Auto-Enrollment & Payment

Knowledge

Knowledge

Knowledge

Eligibility Levels

Tracking/ Notificati

on

Background - Implementing Changes and Greater Challenges Realizing Health Outcome ROI

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

Plan First Enrollment – Medicaid Family Planning

Women

Men

Total

Determining

ROI For Plan

First Project

Project

Implementation

Costs

New Staffing

Costs

Marketing

Costs

Increased

Operational

Time on

Enrollment &

Billing Staff

Initial PI Teams

Costs DMAS, DSS & Local Health District Staff

Hired 2 FTEs to Run Program

Marketing Outreach, Focus

Groups, etc

Additional Time, Billing & Enrolling

Step 1: Calculating All Costs Pre-Implementation – Implementation Phases

Step 1: Calculating All Costs Pre-Implementation – Implementation Phases

$-

$10,000.00

$20,000.00

$30,000.00

$40,000.00

$50,000.00

$60,000.00

$70,000.00

$80,000.00

ALEXA

ND

RIA

ALLEG

HA

NY

ARLIN

GT

ON

CEN

TR

AL S

HEN

AN

DO

AH

CEN

TR

AL V

IRG

INIA

CH

ESA

PEA

KE

CH

EST

ER

FIE

LD

CH

ICK

AH

OM

INY

CRA

TER

CU

MBER

LA

ND

PLA

TEA

U

EA

STER

N S

HO

RE

HA

MPTO

N

LEN

OW

ISC

O

LO

RD

FA

IRFA

X

LO

UD

OU

N

MO

UN

T R

OG

ER

S

NEW

RIV

ER

NO

RFO

LK

PEN

INSU

LA

PIE

DM

ON

T

PIT

TSYLV

AN

IA/D

AN

VIL

LE

PO

RTSM

OU

TH

PR

INC

E W

ILLIA

M

RA

PPA

HA

NN

OC

K

RA

PPA

HA

NN

OC

K-RA

PID

AN

RIC

HM

ON

D

RO

AN

OK

E

SO

UT

HSID

E

TH

OM

AS JEFFER

SO

N

TH

REE R

IVERS

VIR

GIN

IA B

EA

CH

WEST

PIE

DM

ON

T

WEST

ER

N T

IDEW

AT

ER

Measuring Increased Revenue from Medicaid Billing

Through May 2013

Total Paid 2011

Total Paid 2012

2013 Claims

Step 2: Measuring Immediate Impact

Initial View of Short Term ROI Calculations Using the Tool

Plan First Population Pregnancy Rate

12.9

5.07

Comparison of Pregnancy Rate Per Thousand

Average Virginia Rate vs. Plan First Population

Step 3. Measuring ROI of Long Term Health Outcome Impact

Step 3. Measuring ROI of Long Term Health Outcome Impact

• Working with DMAS to Determine Cost Per Pregnancy to Medicaid

• Calculating Variation in Pregnancy Rate to Determine Savings

• Calculating ROI Compares All Costs Including Medicaid Payments for Plan First

Traveler

Supervisor

Business Manager

Office Director

Executive Advisor

Deputy Commissioner

Commissioner

Secretary HHS

Time Spent Per Unit

Salary Cost Per Unit

X

Number of TARs

(1000)

X

Example 2: Proactively Determining Is A PI Project Worth It? Automating the VDH Travel Process

= 73K in

Time and Effort

VS.

Estimated $130,000

Programming Costs

Optimistic Future State

T&E $40,293

Travel Cost Increase

From Approval

Delay

+

=

Sample Routine Operating Cost Calculation

TAR Automation ROI

The tool helps you recognize unanticipated costs and can be great for both prospective and retrospective decision making

ROI Analyses Must be Dynamic ◦ PI Projects vary in complexity and metrics

◦ ROI Tools must be flexible ◦ Amortization of project costs can be challenging as change processes fluctuate

Illustrating & Showing ROI is Critical to Drive & Promote Continuous PI

Measuring Health Outcome ROI Remains A Critical Challenge

◦ ROI for public health remains un-quantified for Public Health ◦ A National Standardized Cost Analyses Calculating Cost Per Incident/Unit should be

established

Susan Logan, MS, MPH

Connecticut Department of Public Health

June 10, 2013

Testing the New ROI Tool in Connecticut: A Prospective Look

Improving the Connecticut Department of Public Health Databases

AIM: Make three DPH databases compliant with the CT DPH policy on collecting

sociodemographic data (similar to federal OMB-15 directive in 1997)

Increase the percentage of compliant databases from 4% to 10%

Put a plan in place to modify the remaining databases

Make sure that all newly-designed databases are aligned with the DPH data collection policy.

The modification process for the targeted databases was implemented and this strategy will be followed on a continuous quality improvement basis through 2014.

Background on Quality Improvement Initiative

6/10/2013 48

Walk-through of

ROI Tool and Review Definitions

Time Periods:

Pre-Implementation

(Plan), Implementation (Do and Check),

Post-implementation

(Act)

Gather Financial Data:

Salaries, Computer

costs, Supply costs, Travel,

etc.

Calculate Costs of Planning

and Intervention

Calculate Costs of

Operations

Calculate Costs/

Savings for Outcomes

ROI is Produced Based on

Prior Calculations

Process

6/10/2013 49

Calculate Costs of QI Planning

Calculate Costs of QI Intervention

Determining Investment Costs: QI Planning and Implementation

• Pre-implementation/Baseline Period • Project team mid-point annual

salaries x FTE spent (plus Fringe) • Include employees assisting with

planning • Office operations costs (printing,

supplies) • Travel for grant award

• Implementation and Post-Implementation Periods

• Project team mid-point annual salaries x FTE spent (plus Fringe) • Include employees assisting with

implementation • Cost of contracting out services • Office operations costs (printing) • Travel for grant award • Modifications to Databases • Communications (storyboards,

easels, DPH Newsletter)

6/10/2013 50

Approximately how many meetings did you attend regarding this QI project? ________

Approximately how much time did you spend on the following activities related to this

QI project?

Attending formal or informal meetings? hours

Making changes to data collection forms? hours

Making changes to databases? hours

Generating cost estimates for changes? hours

Other activities related to this QI project? hours

Please briefly describe these activities: ________________________________________________________________________________________________________________________________________

Customer Satisfaction Survey on QI Process

6/10/2013 51

Collected Data on Time Spent on Planning and Implementation Activities:

Salary Costs for QI Initiative Team

6/10/2013 52

Pre-Implementation - Baseline Implementation Period 1

From To From To


Salary and

Fringe

Salary and

Fringe

Personnel (see below)

List by name or by

category FTE % Cost FTE % Cost

Project Sponsor - Epidemiologist 4

Project team member

5.00% 87,931.00 $ 1,721.98 3.00%

87,931.00 $ 725.43

Project leader-Epidemiologist 2 20.00% 53,143.00 $ 4,162.87 20.00%

53,143.00 $ 2,922.87

Epidemiologist 4 Project team member 4.00% 87,931.00 $ 1,377.59 2.00%

87,931.00 $ 483.62

Epidemiologist 3 Project team member 6.00% 76,884.00 $ 1,806.77 5.00%

76,884.00 $ 1,057.16

Section Chief: Practitioner Licensing Project team member 5.00% 100,000.00 $ 1,958.33 5.00%

100,000.00 $ 1,375.00

Manager Vital Statistics Project team member 5.00% 86,300.00 $ 1,690.04 5.00%

86,300.00 $ 1,186.63

Section Chief: IT Project team member 4.00% 100,000.00 $ 1,566.67 2.00%

100,000.00 $ 550.00

Takes into account time prds (4.7 mos baseline)

Salary Costs for QI Initiative Team: Implementation Prds 2 and 3

6/10/2013 53

Implementation Period 2 Implementation Period 3

From To From To

Cost Category 12/1/12 12/31/13 1/1/14 12/31/14

Salary and

Fringe

Salary and

Fringe

Personnel (see below) FTE % Cost FTE % Cost

Project Sponsor - Epidemiologist 4 6.00% 87,931.00 $ 5,715.52 6.00%

87,931.00 $ 5,275.86

Project leader-Epidemiologist 2 8.00% 53,143.00 $ 4,605.73 8.00%

53,143.00 $ 4,251.44

Epidemiologist 4 4.00% 87,931.00 $ 3,810.34 4.00%

87,931.00 $ 3,517.24

Epidemiologist 3 $ - $ -

Section Chief: Practitioner Licensing $ - $ -

Manager Vital Statistics 3.00% 86,300.00 $ 2,804.75 3.00%

86,300.00 $ 2,589.00

Section Chief: IT 3.00% 100,000.00 $ 3,250.00 3.00%

100,000.00 $ 3,000.00

Data quality committee reviewing databases and overseeing fixes (Impl Prds 2 - 3)

Non-Personnel Investment Costs: Planning and Implementation Periods

6/10/2013 54

Baseline - Pre-implementation Implementation - Year 1

From To From To


Contracted services3

Other contracted services Modifications done - 5 hours at $40.00/hr - 200.00

Office operations4

Printing paper ($5/ream) and toner($602/crtrdg) 303.50 303.50

Office supplies/postage

flip charts (2 cartons at $41/crtn), pens (4 dzn at $11.82/dzn), markers

(4 at $2 ea), post-its ($15/18 pk) 76.26 76.26

Travel/conferencing

Dinners ($26 per) and Taxis( $25 per): Travel to Portland, OR in Apr.

and Charlotte, NC in Dec. 1,050.00 1,050.00

Training - 137.08

Other office operations frames for certificates of recognition/appreciation (11 at $11.70/ frame) 128.70

Communications 8 hours of writing article for DPH newsletter and creating story board - -

Equipment6

Subtotal direct costs $ 17,815.61 $ 17,466.42

Indirect cost percentage (if %

of direct costs)9

Indirect costs9 $ - $ -

Total costs of investment in

strategy $ 17,815.61 $ 17,466.42

Years of expected utility from pre-implementation investments (for amortization)10 1.00

Amortized Pre-Implementation

Investment Costs $17,815.61

Calculate Costs/Savings for Outcomes

ROI is Produced based on Prior Calculations

Determining Costs and Savings for Outcomes and Outputs: ROI Produced

• Baseline and Post Periods • Measures of Production Time • Time to analyze data with

standardized sociodemographic categories (expected reduction)

• Time to review databases (improves over time)

• Improve Health Outcomes: Based on finding new cases in sociodemographic groups

• Taking into account the Investment and Routine Operating Costs Only o Implementation Period 1

Benefit (savings) was 64 cents for every dollar spent

o Implementation Period 2 Cost was $2.21 for every dollar

spent • Adding in Outcomes

o Implementation Period 1 and Overall Savings was $17.05 ($18.16

overall) for every dollar spent

6/10/2013 55

Outcome/Output Measures: Production Time Costs and Savings

6/10/2013 56

Measures of Production Time Baseline Post Period - Year I

TRUE Description: output/event and

time frame

Total

events

Average

time

(days)

Total

time

(days

)

Total

event

s

Averag

e time

(days)

Total

time

(days)

Time to analyze data with standardized

sociodemographic categories

May see reductions in Post Periods-Year

2 and 3 when employees are trained and

databases were starting to be used for

quarterly and final reports: 1/1/2013 to

12/31/2014

7.00

3.00

21.00

7.00

2.50

17.50

Number of databases reviewed by project

sponsor and leader

Spent 1 hour per database review in

baseline and QA checks

8.00

0.20

1.60

8.00

0.03

0.24

Number of databases reviewed by data

quality committee

Will spend 1 hour per database review in

baseline and QA checks

-

-

Total 5 4.52 22.60 5 3.548 17.74

Outcome/Output Measures: Reach Outcomes: Rates

6/10/2013 57

Measures of Reach/Intermediate

Outcomes Baseline Post Period - Year I

FALSE

Description: target population

and outcome Numerato

r

Denominato

r Rate

Numerato

r

Denomin

ator Rate

Finding cancer cases based on self-

identified gender

Using SEER cancer registry as source

data 343 75,250 455.81 350 75,250 465.12

Finding cases of reportable disease

based on sociodemographic

characteristics (e.g. black race)

Focusing program services for African-

Americans throughout the CT 5,000 660,000 757.58 5,500 660,000 833.33

Finding cases of adult lead poisoning

based on sociodemographic

characteristics (e.g. Hispanic) Finding more cases of hispanic origin 22 874,500 2.52 30 874,500 3.43

Composite reach 5,365 1,609,750 333.28 5,880 1,609,750 365.27

Used these rows as a worksheet for the health outcomes on next slide

Did not include data in ROI calculations (unchecked)

Outcome/Output Measures: Health Outcomes Savings

6/10/2013 58

Include data in ROI calculations (checked)

Measures of Health-Related

Outcomes Baseline Post Period -

Year I

Post Period -

Year 2 Post Period - Year 3

TRUE Description Total Total Total

Total

Est. Cost per

Unit

Outcome**

Preventing cancer deaths

(based on better follow-up)

people who self-identify as

other than male or female

Can prevent 5% of cancer

deaths if invasive cancer

found and reported 17 18 18 19 -$50,000.00

Preventing readmissions for

reportable diseases in blacks

Can reduce readmissions

by 10% 500 550 600 660 -$9,500.00

Preventing further cases of

adult lead poisoning in

Hispanics in

neighborhoods/communities

For every one person

exposed and

poisoned,there are 2

others exposed and

potentially poisoned 44 60 70 80 -$1,000.00

Composite outcomes 561 628 688 759

Total value -$5,644,000 -$6,185,000 -$6,670,000 -$7,300,000

Return on Investment Analysis

6/10/2013 59

II. Analysis INCLUDING Output and Outcome Measures (Production Time, Reach, and Health-Related Outcome

Factors)

Implementation Period Total



Amortized Pre-Implementation Costs $17,815.61

Ongoing Implementation Costs $0.00 $14,766.42 $22,158.71 $20,256.01

Total Annual Implementation Costs $17,815.61 $14,766.42 $22,158.71 $20,256.01

x Present Value Factors (see below) $1.00 $0.97 $0.94 $0.92

Total Discounted Annual Implementation Costs $17,815.61 $14,336.33 $20,886.71 $18,537.11 $71,575.76


Estimated Changes in Operating Costs $ (21,330.43) $ 146,265.80 $ 129,506.18

Estimated Changes in Revenue $ - $ - $ -

Total Net Financial Gain/Loss $ 21,330.43 $ (146,265.80) $ (129,506.18)

x Present Value Factors (see below) $ 0.97 $ 0.94 $ 0.92

Total Discounted Annual Net Gain/Loss $ 20,709.15 $ (137,869.54) $ (118,516.50) $ (235,676.89)

Non-Financial Gains/Losses Attributable to Project

Estimated Production Time Gain/Loss 4.86 (28.85) (20.40)

Monetary Value of Time Gain/Loss $ 2,443.55 $ (16,117.17) $ (11,396.54)


Total Discounted Value of Time Gain/Loss $ 2,372.38 $ (15,191.98) $ (10,429.45) $ (23,249.05)

Cumulative ROI including Time Cost $ 0.72 $ (2.45) $ (3.62) $ (3.62)

Estimated Changes in Health-Related Outcomes 67.00 127.00 198.00

Estimated Cost per 1 Unit Improvement in Outcomes $ (95.12) $ 1,250.05 $ 692.19

Monetary Value of Outcome Improvement/Loss $ 541,000.00 $ 485,000.00 $ 630,000.00


Total Discounted Value of Outcome Improvement/Loss $ 525,242.72 $ 457,159.02 $ 576,539.25 $ 1,558,940.98

Cumulative ROI including Value of Health-Related Outcomes $ 16.98 $ 16.31 $ 18.49 $ 18.49

Cumulative ROI Including All Non-Financial Gains/Losses $ 17.05 $ 16.07 $ 18.16 $ 18.16

Used prospectively in this case

Seemed daunting at first, but once data is collected, calculations are done for you

Used customer satisfaction survey to gather data on time spent on planning and making modifications to databases

Customized to fit our needs

Highlights of Using the ROI Tool for a Connecticut DPH QI Project

6/10/2013 60

Can be applied for many different types of QI projects

Data quality projects may not have a large return, but it is the “right thing to do”

Projects that improve efficiency and reduce process steps/time will see a greater return on investment

Future Use of the ROI Tool:

ASTHO National Demonstration Project (3/1/12 – 10/31/13)

Make data accessible to local health departments – reduce time for state DPH staff to handle data requests

Save time cleaning up data

Applications of the ROI Tool

6/10/2013 61

Thank you

6/19/2013 62

estimating return on investment for public health improvements

Documents