practical strategies for learning evaluation – metrics that matter in identity & passport...
TRANSCRIPT
![Page 1: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/1.jpg)
Practical strategies for Learning Evaluation – Metrics That Matter in
Identity & Passport Service (IPS)
Prepared by : Andrew Somiah
Systems Training Manager
Date 18th July 2011
Version 2.0
Not protectively marked
1
![Page 2: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/2.jpg)
Overview of IPS
Identity and Passport Service (IPS) is a Public Sector
Organisation
Approximately 3800 employees spread across the
UK
Core Business Products:
─ Passports & Identity Management
─ Birth, Marriage, Death & Civil Partnership
Certificates
2
![Page 3: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/3.jpg)
Overview of Learning & Development Function in IPS
Centralised Learning & Development function with a
wide geographical remit.
Classroom trainer-led events
E-learning delivered by Kallidus Learning
Management System (LMS)
Blended learning solutions managed via the Learning
Management System (LMS).
Approximately 589.6 days of training delivered for
2009-2010 financial year
![Page 4: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/4.jpg)
5 Levels of Evaluation Model used in IPS (Jack Philips Model)
![Page 5: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/5.jpg)
Learning Measurement & MTM Strategy
• MTM is integrated with the IPS LMS– Estimate, Isolate & Adjust technique used by MTM
• All training courses including some e-learning are evaluated
• Internal targets and an Organisational Development Score Card has been set to drive strategy
• Three key indicators that are monitored monthly:
– Trainer Performance
– ROI: Cost Benefit Ratio
– Learning Impact & Value (business results)
• IPS targets are incorporated into KPOs for L&D staff who are performance managed with these
5
![Page 6: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/6.jpg)
Learning Measurement & MTM Strategy
Implementing
Tactical
Eff
ectiv
enes
s
Time
IPS-Strategic
Optimized
1
2
3
4
5
Collecting evaluation data and measure business results
LMS dashboards presentation to business
Presenting monthly analyses to business
Comprehensive
6
![Page 7: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/7.jpg)
1st Quarter of 2010 Figures
April May June
No of Events 85 84 90
No of Delegates 322 395 650
Training days 59.9 79.9 38.5
Average delivery per trainer (Target: 100 days per yr i.e. 9.0 average per month)
2.7 6.3 5.4
Cost benefit Ratio (Target: £1:3.00)
£ 1:1.74 £ 1:1.86 £ 1:2.44
Average Trainer Score (Target: 6.25)
5.41 6.37 6.43
Evaluation Returns (Target: 65%)
52.02% 58.08% 56.78%
7
![Page 8: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/8.jpg)
Impact on Business
• To improve impact and value to business: information is displayed on the LMS dashboard
8
![Page 9: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/9.jpg)
Practical Approach to Data Analysis
9
Evaluation Data Analysis
![Page 10: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/10.jpg)
Review & Analysis Cycle
10
![Page 11: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/11.jpg)
• Human Capital ROI Card
• Business Results ROI Card
• Instructor Performance Summary Report
• Training Improvement Summary Report by Trainers
• Project Initiation Documents/Action plan by Training Managers
• Course Effectiveness Summary Report (Director & board Level)
• Talent Development Executive Report (Director & board Level)
11
Reports Used for Practical Analysis
![Page 12: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/12.jpg)
Challenges & Tips
• There was initial resistance when the system was implemented as it was a cultural change
• IPS has put in place an Organisational Development balance score card accepted at board level
• The internal benchmark targets have been incorporated into the organisations Key Performance Indicators
• Align evaluation strategy to business objectives
•Note: Its is a Process not a project expect results over time.
• Simple continuous marketing of the benefits to business units helps improve evaluation return rates
12
![Page 13: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/13.jpg)
Summary
• Aspirations- Evaluation to become second nature and part of the culture
• MTM is providing an invaluable resource for:
– demonstrating the worth of training to organisation
– provides performance information for making crucial strategic decisions
– Could have a positive impact on the bottom line of any organisation
![Page 14: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/14.jpg)
Questions
![Page 15: Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f005503460f94c15ee7/html5/thumbnails/15.jpg)
Contact and Further Information
Andrew Somiah
Systems Training Manager
Identity and Passport Service
HQ Learning & Development
4th Floor Mailpoint B NW Peel Building
2 Marsham Street
London
SW1P 4DF
T: 0207 0356 798
Stuart Halliday
EMEA Sales Director
KnowledgeAdvisors
P: +44 (0) 203 371 1165
M: +44 (0) 771 7507 945