Workshop EvaluationWorth the Effort!

NLTAPA ConferenceBoise, ID

Evaluation Overview

• Purpose: To determine the effectiveness of a training program.

Evaluation Overview

• Four-level model developed by Donald Kirkpatrick (1998).

• Reaction• Learning• Behavior• Results

Evaluation Overview (continued)

• Each successive evaluation level is built on information provided by the lower level.

Level 1

Level 2

Level 3

Level 4

Reactio

n – Learning –

Beha

vior – Re

sults –

Reaction (Level 1)

What they thought and felt about the training.

Level 1

Level 1

Courses expanded, or condensed Instructors adjusted Courses removed from the program pending

further review

Courses expanded, or condensedStormwater Managemen

t

Participant Comments (Fall 2010)

Too long and too deep of

explanations.

Too much info that doesn’t concern our

needs.

Maintenance section is too

repetitive.

Instructors Adjusted

Winter Maintenance

Participant Comments (Fall 2012)

• Instructor should be someone that worked in this field at public works level.

• Instructor seems to lack practical experience in this field.

• Less talk about cost. More on winter maintenance. (That's what our bosses are for– keeping cost down.)

Courses Removed

Traffic Signals Participant Comments

(Fall 2012)

• Make half days.• Break into different sections.• Break into different levels –

decision planning vs. on-going maintenance.

Learning – Level 2

Pre and Post Quiz

• Developed to evaluate the instruction and content of our training programs

(NOT the student)

• Consists of 6 questions• Note your answers on the back of the evaluation

sheet

Quiz Example Question

1. Denying physical access to someone because of their impairment is:

a. Sometimes necessaryb. Discriminationc. Permitted by law in certain circumstancesd. Unimportant compared to making roads safe for cars

Learning

Course content modified Quiz questions refined

“Un-Learning” in Winter Maintenance

Behavior - Level 3

Enhanced skills and knowledge identified on the jobo Newer processo 6 month evaluation

Level 3

Results – Level 4

Cost savings of approximately $5,000 related to remediation of current signage

$1,500 of savings in equipment expenditures

$30,000 of expenses avoided that might have been incurred if gas line had to be relocated after completion

Lawsuits avoided

Questions ...

LTAP – Local Technical Assistance ProgramPennsylvania Department of TransportationBureau of Planning and Research400 North Street, 6th FloorHarrisburg, PA 17120

• E-mail: LTAP@state.pa.us• Web Site: www.ltap.state.pa.us