Unit 10: Evaluating Training and Return on Investment

2009

2

Unit 10, Class 1: Evaluating Training

• Learning ObjectivesBy the end of this unit, students will:> Determine benefits of a training program.> Calculate benefit/cost ratio.> Calculate return on investment (ROI). > Identify when ROI evaluation is not

appropriate.> Use other methods to verify training value

when ROI is not appropriate.

©SHRM 2009

Training Evaluation

• Reluctance to evaluate:> Managers are unwilling to devote time to evaluation.> Lack of know-how or no importance.

• Why evaluate?> Link to organizational strategy.> Effectiveness of training.> Return on investment.

3©SHRM 2009

Training Evaluation

Was the training effective?

• Training effectiveness: The benefits the organization and trainees receive from training.

4©SHRM 2009

Training Evaluation

• Formative evaluation: Evaluation of training that takes place during program design. > May result in content change.> May involve pilot test.> May adjust to meet needs of the trainees.

5©SHRM 2009

Training Evaluation

• Summative evaluation: Evaluation conducted at the end of training.> Used to determine the extent to which

trainees have changed as a result of the training program.

> Used to measure return on investment.

6©SHRM 2009

Instructional Design: ADDIE Model

7©SHRM 2009

Evaluation Process

• Conduct a needs analysis.• Develop measurable learning outcomes and

plan for transfer of training.• Develop outcome measures.• Choose an evaluation strategy.• Plan and execute the evaluation.

8©SHRM 2009

Kirkpatrick’s Four-Level Model of Evaluation

• Level 1: Reaction• Level 2: Learning• Level 3: Behavior• Level 4: Results

9©SHRM 2009

Level 1: Reaction

• Reaction: How did participants react to the program?> Smile sheets.> Informal comments from participants.> Focus group sessions with participants.

10©SHRM 2009

Level 2: Learning

• Learning: To what extent did participants improve knowledge and skills and change attitudes as a result of the training?> Pre- and post-tests scores.> On-the-job assessment.> Supervisor reports.

11©SHRM 2009

Level 3: Behavior

• Behavior: Do learners use their newly acquired skills and knowledge on the job?> Transfer of training.> On-the-job observation.> Self-evaluation.> Supervisor and peer evaluation.

12©SHRM 2009

Level 4: Results

• Results: What organizational benefits resulted from the training?> Difficult and costly to collect.> Impossible to isolate the results of training.> Measuring return on investment.> Financial reports.> Quality inspections.> Interviews.

13©SHRM 2009

Levels of Evaluation vs. Value

14

Kirkpatrick, & Kirkpatrick, 2006

©SHRM 2009

Your Training Project

Design evaluation instruments for your training project.

15©SHRM 2009

Unit 10, Class 2: Return on Investment: Benefit-Cost Ratio

• Benefit-Cost Ratio:> Aids in decision-making process.> Consistent analysis across programs.> Information difficult to obtain.

©SHRM 200816

Return on Investment

• Return on Investment/Benefit-Cost Ratio:

Program Benefits

Benefit-Cost Ratio = ----------------------------

Program Costs

$2,500

Benefit-Cost = 2.5:1 -----------

$1,000

17©SHRM 2009

Benefit-Cost Ratio

Program benefits : $6,500

Program costs: $8,495

What is the benefit-cost ratio?

18©SHRM 2009

What About ROI?

Return on Investment – ROI (%)

Program benefit: $2,500

----------- = 2.5 x 100 = 250%

Program cost: $1,000

ROI = 250%

19©SHRM 2009

Determining Benefits

• Measuring training benefits:> Benefits must consider training objectives.> Literature summaries of benefits of specific training.> Assessment of pilot training programs.> Observations of successful trainees.> Estimates from trainees and managers.

20©SHRM 2009

Programs Best Suited for ROI Analysis

• Training appropriate for ROI analysis:> Clearly identified outcomes.> Not one-time events.> Broad-based and highly visible in the organization.> Strategically focused.> Training effects can be isolated.

21©SHRM 2009

When ROI Isn’t Appropriate

• Justifying training when ROI isn’t the answer:> Success cases.> Measuring the payback period.> The consequences of NOT training.

• Focus on most important programs. • Make training a true business partnership.

22©SHRM 2009

Training and Organization Success

23

Strategic Planning

Assessment

Design

Development

Implementation

Evaluation