Evaluating Training EffortOrganisations are under pressure to justify
various expenses.
Business heads and training managers are under pressure to prove the effectiveness of training.
Two basic issues to address when evaluating:-
How to design the evaluation study? What to measure?
www.a2zmba.com
Measuring Training Effectiveness Basically three things that we can measure:
Participants’ reactions to the program;
What the trainees learned from the program;
To what extent on-the-job behavior changed.
www.a2zmba.com
Designing the Evaluation StudyTime Series:- Series of measures
before and after the training program.
Controlled Experimentation:-
Formal methods for testing the effectiveness of a training program, preferably with before-and-after tests and a control group.
www.a2zmba.com
What to Measure? Kirkpatrick’s TaxonomyReactions: Through “ Happy
sheets questionnaire” or “360-degree questionnaire”.
Learning
Behavior
Results
www.a2zmba.com
360 Degree Questionnaire Benefits of using a 360-degree format include:
Workplace behaviour improvements
The pre-program assessment can be used by the participant as a focus for their learning.
The participant is given a much broader appreciation of the application of their behaviours.
The reporting after the post assessment allows the individual to see how they have grown or developed as a result of the training program.
It provides the facilitator with hard data for proving the cost-effectiveness of the training intervention to Management.
www.a2zmba.com
Issues against
The Kirkpatrick framework represents a trainer’s notion of what constitutes effective evaluation, not a business manager’s.
The second issue deals with hierarchy, the assumption that
each level has more value than the previous one.
The third issue deals with causality, the assumption that each level is caused by the previous one.
Kirkpatrick’s taxonomic model is conceptually flawed as it does not fully identify all the constructs underlying the phenomena of interest. Furthermore, the results provided by the model can lead to incorrect decisions since major intervening variables are not included.
www.a2zmba.com
Other Approaches….QAI India QAI is a leading global consulting
organization (Asia’s largest and world’s third largest organization) addressing ‘Operational Excellence' in IT, BPO and Knowledge intensive organizations.
Effectiveness of a training program is measured at 5 levels:
I. Need for trainingII. Satisfaction of participantsIII. Learning at the classroomIV. Change in BehaviorV. Improvement in results
www.a2zmba.com
ROI approach by Jack PhillipsROI = Net Program Benefits X 100 % Program Costs
ROI couches results and expectations in the language of business and is often a hurdle
a project must clear in order to secure funding. Given that learning is
increasingly perceived as an investment by the organization, it seems only natural
that it should be evaluated in the same way as other large investments, in terms
of costs and benefits
www.a2zmba.com
Software Companies:
They use Heuristic Evaluation Quality Score (HEQS)
Steps of HEQS are:
www.a2zmba.com
1. Identify the application, parameters and the severity ratings
2. Standardize the scope of evaluation
3. Provide a knowledge transfer (optional)
4. Standardize the time for evaluation
5. Standardize the evaluation format
6. Proceed to evaluate
7. Use the individual evaluations to arrive at a benchmark
8. Use this benchmark to measure the skills of each evaluator
9. Derive Insights
www.a2zmba.com
www.a2zmba.com
• The IBM Certified Trainer (ICT) program is a comprehensive program for Trainers' Preparation and Certification.
• Validates skills using three proficiency levels: IBM Authorized to Teach. (IAT) IBM Professional Trainer (IPT) IBM Training Expert (ITE)
• E.g. Blended learning programs encompasses multiple learning strategies and delivery formats, ranging from those that are solely technology-based
to those that support interactions between people. Training effectiveness is measured by a control group of employees who receive no training. Provides real-world-based assessments with
feedback• Increase in its annual investment in learning programs that are
blended (with increases between 30 and 40 percent each year for the past four years).
www.a2zmba.com
Management Development Programmes(MDPs) have been institutionalised.
Wipro has different programmes catering to different life-cycle stages of the manager.
Different people may get different amounts of training at different times.
360 degree feedback.Training Quality measured against six sigma.
Manufacturing CompanyFeatherlite IndiaTraining given on various
topicsTraining conducted every 3
months or 6 months depending on requirements
In-house and external trainers
Happy sheet Questionnaire known as “ Survey Forms”
Monthly Performance Appraisal
Interview Rating Sheets
www.a2zmba.com