m101 and tu101 evaluating training effectiveness: the ... · evaluating training programs 2. expand...
Post on 14-Oct-2020
7 Views
Preview:
TRANSCRIPT
M101 and TU101Evaluating Training Effectiveness:
The Kirkpatrick Four LevelsTM
Dr. Donald L. Kirkpatrick
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Trademark and Copyright Warning© Copyright 2010-2011, Kirkpatrick Partners, LLC. All
rights reserved. This presentation remains the exclusive property of Kirkpatrick Partners, LLC. Distribution or duplication of these slides and content, in whole or in part, in any way is strictly prohibited. These are for the personal use of program attendees only.
The following marks are the property of Kirkpatrick Partners, LLC as related to educational / training materials, programs and books: Kirkpatrick®
Kirkpatrick Four LevelsTM
The One and Only Kirkpatrick®
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
The Trainer’s Song
Words by Donald KirkpatrickTune: Clementine
They’re the greatest in the world,There is nothing more to say,No one dares to contradict it,They’re the Packers from Green Bay.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
The Trainer’s Song
There were anti-Packer people.To run them down was just their goal.Now they’re eating crow and crying, ‘Cause they won the SUPER BOWL!
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
The Trainer’s Song
We are trainers teaching “teamwork”,We need a guide to show the way,Let us follow the example,Of the Packers from Green Bay!
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
HRD is:
A SCIENCE– Concepts– Theories– Principles– Techniques
AN ART– Application to your situation
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Teaching Approach
SMART
vs.
PIE
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Quiz1-4. In sequence, what are the 4 levels?5. What do we call a test we administer prior to a training
program?6. What do we call a test we administer after a training
program?7. What do we call the group that receives the training?8. What do we call the group that doesn’t?9. T or F We can borrow results from other organizations
if the program is the same.10. T or F Those who plan, coordinate, and teach training
programs are better able to judge its effectiveness than those who attend.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Requirements for an Effective Program
1. Based on __ __ __ __ __.2. Aimed at __ __ __ __ __ __ __ __ __ __3. Scheduled at the right __ __ __ __.4. Held at the right __ __ __ __ __.5. For the right __ __ __ __ __ __.6. Conducted by an effective __ __ __ __ __ __.7. Using effective __ __ __ __ __ __ __ __ __ __.8. __ __ __ __ __ __ __ __ __ __ are reached.9. Participants are __ __ __ __ __ __ __ __ __.10. Program is __ __ __ __ __ __ __ __ __.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Requirements for an Effective Program
1. Based on needs.2. Aimed at objectives.3. Scheduled at the right time.4. Held at the right place.5. For the right people.6. Conducted by an effective leader.7. Using effective techniques.8. Objectives are reached.9. Participants are satisfied.10. Program is evaluated.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Workshop Objectives
• To Understand– Why Evaluate?– The Four Levels– Guidelines– Forms and Approaches
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Why Evaluate?
• Should the program be continued?
• How can the program be improved?
• How can trainers justify their existence?
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Level 1: REACTIONHow participants react to the training
Kirkpatrick Model
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Level 2: LEARNINGThe extent to which participants acquire intended knowledge, skills, and attitudes
Kirkpatrick Model
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Level 3: BEHAVIOR The extent to which participants apply what
they learn on the job
Kirkpatrick Model
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Level 4: RESULTS The final outcomes that occur as a result of the
training and reinforcement
Kirkpatrick Model
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Evaluation Principles
1. Evaluation methods and forms can be borrowed from other organizations and used or adapted to your situation.
2. Evaluation results cannot be borrowed even if the program is the same.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Guidelines for Evaluating Reaction
• Determine what you want to find out.• Design a form that will quantify reactions.• Encourage written comments.• Get 100% immediate response.• If desirable, get delayed reactions.• Develop acceptable standards.• Measure future reactions against the
standard.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Methods for Evaluating Reaction
• Reaction sheet• Trained observer• Focus group
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Reaction Sheet #1 Instructions: Please complete the form by checking theappropriate answer for each item:SD – strongly disagree D – disagree N – neutral A – agree SA – strongly disagree DNA – does not apply
SD D N A SA DNA1. The meeting started on time. 2. The program was practical and met
my needs. 3. The program maintained my interest
and attention. 4. The objectives were clearly stated.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Reaction Sheet #2 Instructions: Select your response after the
following items. 21. The length of the meeting was: ____ Too long ____ Just right ____ Too short
22. I am clear on how to apply what I learned: ____ very clear ____ clear ____ quite clear ____ not clear
23. What are three things you learned during the program?
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Guidelines for Evaluating Learning
• Measure before and after attitudes, knowledge, and/or skills.
• Use a paper and pencil test for knowledge and attitudes.
• Use a performance test for skills.• Use a focus group for Levels 1 and 2
combined.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Methods for Evaluating Learning
• Pre and post test• Performance observation
– Teach back– Role play
• Focus group
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Selected Items from “Management Inventory on Time
Management”A DA 1. Managers should be rewarded on the basis of the
amount of time they devote to the organization.A DA 2. It’s a good idea to handle each email or piece of
paper only once.A DA 3. It’s a good idea to do two things at once in order to save time.A DA 4. Unwanted visitors should be treated ‘coolly’ so they won’t
stay.A DA 5. When a direct report comes to you with a problem
which you can’t immediately solve, it’s a good policy to say, “I’ll look into it and get back to you”
A DA 6. Delegation should always include the results to be achieved as well as the activities to be performed.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
FormPlease circle the items that you answered “incorrectly”according to the scoring key. Then, subtract the number wrong from 45 to get your score.
1 2 3 4 5 6 7 8 910 11 12 13 14 15 16 17 1819 20 21 22 23 24 25 26 2728 29 30 31 32 33 34 35 3637 38 39 40 41 42 43 44 45
Your Score 45 - ___ = ___
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Response to MITM Item 4“Unwanted visitors should be treated “coolly”
so they don’t stay long.”
Experimental Group Control GroupAgree Disagree Agree Disagree
Pretest 5 20 5 20
Posttest 10 15 6 19
Gain +5 +1
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Response to MITM Item 6“Delegation should always include the results to beachieved as well as the activities to be performed.”
Experimental Group Control GroupAgree Disagree Agree Disagree
Pretest 7 18 6 19
Posttest 23 2 7 18
Gain +16 +1
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Decision MakingManager
QUALITY Subordinates ACCEPTANCE
Past FutureAvg Range Avg Range25% 10-50% 16% 5-40% 1. Manager decides and sells decision37% 10-50% 30% 10-50% 2. Manager asks for input, considers it,
decides, and sells decision22% 10-50% 25% 5-50% 3. Manager leads subordinates in making
a “consensus” decision16% 0-40% 29% 0-60% 4. Manager empowers subordinates to ____ ____ make the decision100% 100%
ServiceMaster Executives August, 1996
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Transferring Learning to Sustainable Behaviors
Give a man a fish and he
will eat today. Teach him how to fish
and he will eat forever
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Change in Behavior Requires:
1. Desire to Change2. Necessary Knowledge and Skills3. The “Right” Job Climate4. Encouragement and Help5. Rewards for Change
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
The Job ClimateSupervisor Department Head
CLASSROOMKnowledgeSkillsAttitudes
CLIMATEPreventingDiscouragingNeutralEncouragingRequiring
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Guidelines for Evaluating Behavior
• Measure on a before / after basis if practical.
• Allow time for behavior change to take place (perhaps 3-6 months).
• Get 100% response or a sampling.• Repeat at appropriate times.• Consider cost vs. benefits.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Methods for Evaluating Behaviors
• Checklists and observation• Surveys• Interviews (trainee, boss, co-workers,
direct reports)
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Behavior Change(3-6 months after program)
Survey or Patterned Interview1. Explain purpose of survey/interview.2. Review program content.3. Ask the program participant:
“To what extent have you changed behavior as a result of the program you attended?”__ Large extent __ Somewhat __ Not at all
If “Large extent” or “somewhat”, please explain:
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Behavior Change(3-6 months after program)
4. If “not at all”, please indicate why not:___ Program content wasn’t practical___ No opportunity to use what I learned___ My boss prevented or discouraged me to change___ Other higher priorities___ Other reason (please explain)
5. In the future, to what extent do you plan to change your behavior?
___ Large extent ___ Somewhat ___ Not at allPlease explain:
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Guidelines for Evaluating Results
• Measure on a before and after basis. • Allow time for possible results to take place
(possibly 6-12 months). • Repeat at appropriate times. • Use a control group if practical. • Consider short and long term metrics. • Consider “evidence” if “proof” is not possible.
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Methods for Evaluating Results
• Business / HR metrics• Survey (manager)• Interview (manager)
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Reducing TurnoverScenario:It was determined that the main reason for the high turnover was the
poor way supervisors inducted and trained new employees.A training program was conducted in April on “How to Induct and Train
New Employees”Goal:Reduce monthly turnover to 2% or less
Employee Turnover FiguresTraining program held
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec4% 5% 6% 5% 2% 2% 1% 2% 2% 1% 1% 2%
Proof or evidence?
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Increased SalesIncrease in sales for 3 month period over the same
period last year.
Experimental ControlRegions Regions Difference $987,885 ($356,920) $1,344,805
J.P. Huller, Sales Training Manager, Hobart Corporation, Troy, Ohio
Proof or Evidence?
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Suggestions for Evaluating Training Programs
1. Measure reaction to all programs• Develop your own form• Gather data• Establish standards of acceptable performance• Improve programs
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Suggestions for Evaluating Training Programs
2. Expand evaluation to measure:• Learning• Behavior• Results (including ROE)
3. Consider:• Available evaluation resources• Frequency of programs• Cost of programs• Cost of the evaluation• Executive interest / demand
© 2009- 2011, Kirkpatrick Partners, LLC. All rights reserved.
Register With Us for FREE Resources
Links to:• Online resource library with 40+ items• White paper and article• LinkedIn Kirkpatrick Evaluation discussion group
Subscription to:• Weekly e-newsletter
– Kirkpatrick Quick Tips– Articles– News, specials and events
Go to kirkpatrickpartners.com
Thank you!
kirkpatrickpartners.cominformation@kirkpatrickpartners.com
(443) 856-4500
top related