Evaluating Training Programs

Level 5: Return on Investment

Kelly Arthur

Richard Gage-Little

Dale Munson

E-Poster Objectives

Define the fifth level of evaluation Explain how to find Return on Investment

(ROI) for training programs Identify benefits and disadvantages

2 of 21

Building on Kirkpatrick’s Four Levels

1. Reaction

2. Learning

3. Behavior

4. Results

5. Return on investment (ROI)

3 of 21

Why Was It Created?

The fifth level was created because Kirkpatrick’s four levels only focus on the effectiveness of training events and not monetary benefits.

Instructional Developers needed a way to demonstrate cost-value and accountability for their training.

4 of 21

What Is the Fifth Level?

Known as ROI, the fifth level was published by Jack Phillips to quantify the monetary value of training investments.

ROI answers the question, “For every dollar invested in training, how many dollars does the employer get back?”

In other words, show me the money.

5 of 21

What Does it Do?

ROI converts Kirkpatrick’s fourth level data (business results) into monetary values and compares it with the cost of training. ROI is only a financial measure of training’s value, it does not provide information to improve training.

ROI% = Benefits – Costs of Training x 100 Costs of Training

6 of 21

Why Should ROI Be Used?

“Many programs have failed to deliver what wasexpected; consequently, program sponsors haverequested ROI calculations/justifications.”

“The concern for accountability in all functions inorganizations is increasing; thus, the training anddevelopment function becomes one of many support efforts under scrutiny.”

Jack Phillips

7 of 21

Isolate training effects

2

Convert data into

$$$

3

Tabulate program

costs

4

CalculateROI

5

Collect data

1

Identifyintangiblebenefits

6

How to Determine ROI (animation)

8 of 21

Step 1: Collect Data

Hard data – objective and easy to measure– Output (units produced, items sold, etc.)

– Quality (scrap, waste, rework, product defects)

– Time (equipment downtime, employee overtime)

Soft data – subjective and difficult to measure– Work habits (employee absenteeism, tardiness)

– Work climate (employee grievances, turnover)

– Attitudes (employee loyalty, self-confidence)

– New skills (decisions made, problems solved)

9 of 21

Step 2: Isolate Training Effects

Because many factors influence performance, it is necessary to determine the amount of improvement that is directly attributable to training.

Example: After attending a training program, sales people sold 20 percent more widgets over the next six months. Was it the training? Or, maybe it was the new reorganization or new incentive program.

10 of 21

Step 3: Convert Data into $$ Values

In this step, level four data are converted into monetary values and compared against program costs.

Level four measurements: – Documented improvements in the standard of work

and/or care delivered to clients.– Service targets being met more effectively.– Work being accepted by others (repeat business,

testimonials, complementary letters).– Greater measurable productivity, improved safety

records, etc.

11 of 21

Step 4: Tabulate Program Costs

Costs can be categorized into three types:

1. One-time costs: – Analysis and design– Program development – Equipment and software

2. Cost per offering: – Facility rental– Instructor's salary

3. Cost per participant:– Travel – Per diem expenses– Tuition

12 of 21

Step 5: Calculate ROI

Formulas

1. ROI = Total Cost of Training Number of Students

2. ROI (%) = Training Costs - Benefits x 100 Training Costs

13 of 21

Step 6: Identify Intangible Benefits

Intangible benefits are activities or conditions that are extremely difficult to quantify.

Examples: Improved teamwork Improved customer service Increased job satisfaction Reduction in customer complaints Improved communications Increased flexibility Employee morale

14 of 21

The ROI Formula in Action

Krispy Kreme’s evaluation research determined that there was a 10% increase in the number of donuts sold after a new selling skills training program was conducted. Other data revealed that each 1% increase in sales is equal to increased annual revenue of $25,000 for a total of $250,000 in benefits. The cost of training was $75,000. What was the ROI?

Benefits - Costs of training x 100 = %ROI Costs of training

15 of 21

Let’s Work it Out (animation)

($250,000 - $75,000)(100)

$75,000

= ($175,000)(100)

$75,000

($17,500,000)

$75,000= 233%

So, for every $1 invested in training, the organization realized a benefit of $2.33.

Benefits - Costs of training x 100 = %ROI Cost of training

16 of 21

Some Benefits of ROI Evaluations

ROI also provides: A comprehensive measure of training programs Comparability between training programs Benefits and cost in percentage terms

17 of 21

Some Limitations of ROI Evaluations

It is estimated that only 14% of organizations actually evaluate ROI. Why? Many companies don’t perform level four evaluations Difficult to isolate training benefits ROI formula is only an estimation

18 of 21

Summary

ROI is the fifth level of evaluation It provides a monetary value for training

investments ROI is only concerned with how much a

training is “worth” The ROI formula can be used to determine if

the benefits of training outweigh the costs of training

ROI is difficult to determine effectively

19 of 21

Helpful URLs

http://www.learnativity.com/roi-learning.html

http://www.performancexpress.org/0205/mainframe0205.html 

http://www1.astd.org/news_letter/October/Links/phillips.html http://www.performancexpress.org/0208/mainframe0208.html 

http://www.arches.uga.edu/~rali/edit8350/ROI/default.htm

20 of 21

References

Long, Larry. (1999). ROI: Capturing the Big Picture. Technical Training. Retrieved November 12, 2002 from the World Wide Web http://www.astd.org/virtual_community/comm_evaluation/long.pdf

Martin, John. (2000). ROI: Do It & Prove It. ASTD. Retrieved November 12, 2002 from the World Wide Web http:www.astd.org/CMS/templates/index.html?template_id=1&articleid=26100

Phillips, Jack. (1996) Measuring ROI: The Fifth Level of Evaluation. Technical & Skills Training. Retrieved November 12, 2002 from the World Wide Web www.astd.org/virtual_community/ comm_evaluation/phillips.pdf

21 of 21