Dr. Irene T. Boland, Chief Learning Officer, Learning Development Institute

Laura Nelson, VP of Food Safety & Global Alliances, Alchemy Systems

Effective Metrics to Advance

Your Food Safety Training

October 4, 2017

• Laura Nelson• VP of Food Safety & Global Alliances

• Dr. Irene T. Boland• Chief Learning Officers

Introduce Speakers

2

Today’s Discussion

3

1. Setting Learning Strategies

2. Setting Operational Metrics to Measure Training Effectiveness Across Diverse Enterprises: Conagra Brand Case Study

3. Resources

4. Q&A

4

Setting Learning Strategies

The Brands that LDI Supports

5

POLL

• I know what works for me – and I use that.

• I know what works because I have trained people.

• I have studied adult learning.

Describe your expertise in adult learning.

6

What is Learning Strategy?

OrganizationStudy Teach

Me My learners Everyone in my company

7

Where to Start?

8

Act

Focus

Scan

Setting Priorities for Food Safety Learning

9

Setting Priorities for Food Safety Learning - Scan

Scan

10

So much to measure…

11

Recalls Customer Complaints

3rd Party Audit Government Inspection

Internal AuditPreventive Controls

GMPs

Chemicals & Toxins

Product Identity

Allergens

LabelingCook Time and Temperature

Adulteration

Transportation

Storage

1. Train the worker

2. Observe the worker

3. Can do the task

4. Audit the worker on the task

FDA Food Safety Modernization Act

12

Scan: Start with Leadership

CEO

Food Safety Exec

Leader Leader

Quality Exec

Leader

13

How might we change?

START STOP

MORE LESS

14

SCAN: Gathering Data

We have 10 plants

15

SCAN: Gathering Data

We have 100 SKUs

16

SCAN: Gathering Data

Representative Sample of Plants

Acquisition

Highest Volume

Most SKUs

Recent Issues

17

Who to send?

The Inquisitor The Friend

18

Q: Training Today

✓ Standard Compliance

✓ Have to

✓ Stay out of trouble

KNOWN

What else?

Why?

Results?

19

EXPLORE

Performance Improvements?

20

Opportunities

21

How might we change?

START STOP

MORE LESS

22

Focus

Setting Priorities for Food Safety Learning - Focus

23

Deciding what to teach

Food Safety

Interesting

Important

Useful

Just in case

ScienceBig picture

One more thing

Related details

Compelling Story

Once a year

24

Cognitive Overload

Cognitive Overload

Cognitive Overload

25

Policy

Procedure

Performance Support

The details that help us get it done

How we will do it

What we will do – and why

Food Safety Modernization Act

26

Performance Support

✓ Standard Operating Procedure

✓ One Point Lesson

✓ Quick Reference Guide

✓ Checklist

✓ Shift Hand Off Meetings

✓ Posters

✓ Video in breakroom

✓ eLearning

✓ Classroom training

✓ Offsite training

✓ Observation

✓ Coaching

Sometimes training is NOT the answer

27

Gap Closures

TODAY TOMORROWACTIONS

Measures:• Attendance• Quiz

Measures to add:• Long term learning• Behavior

performance

• Create checklists of knowledge and what good performance looks like

• Train supervisors to use checklists and provide constructive feedback

Learning Methods:• eLearning• Classroom

Methods to add:• One point lessons• Communication

• Create templates for one point lessons, shift hand-off and posters.

• Train managers and plant leaders to use the templates to create new learning content as needed

28

Focus: Confirm Across All Plants

Did we get it right?

29

Focus: Confirm with Leadership

CEO

Food Safety Exec

Leader Leader

Quality Exec

Leader

30

Act

Setting Priorities for Food Safety Learning – Act

31

POLL

• Attendance or Reaction

• Quiz during training

• Check for learner retention later

• Observe behavior change

• Return on Investment (ROI)

What metrics do you track about Food Safety learning today?

32

Kirkpatrick Model of Assessment

ROI

Behavior Change

Learning

Reaction

Attendance33

Benchmark: What Training Costs

To Build

Subject matter expert hours 80

x Hourly overhead cost of SME $ 80

$ 6,400

To Host

Number of learners 100

Hours of training 6

x Hourly overhead cost of learners

$ 60

$ 36,000

Other Costs

• Travel

• Facility Rental

• Outside Expertise

34

35

Setting Operational Metrics: Conagra Brand Case Study

Contrasting Training Program Maturity

36

✓ Orientation to cover basics

✓ “Buddy” system

✓ “One and Done” training

✓ Inconsistent/outdated training content

✓ Compliant driven training

✓ Limited training metrics

✓ Siloed training programs

✓ Orientation + ongoing monthly training

✓ Blended learning approach including mentoring

✓ Customized training to achieve job competency

✓ “Best-in-Class”, vetted, consistent training

✓ Behavioral driven training

✓ Success metrics tied to operational objectives

✓ Cross-functional training teams

POLL

• Yes, I do.

• No, I do not.

Despite all of our classroom training, I still have employees not following our food safety program.

37

38

“Every day, every shift, every job role,

we have a validated person in the job role”

—Dale Greenbank, Conagra BrandsContinuous Improvement Manager, Rensselaer, IN

39

Continuous Skills Development (CSD) Process Enterprise Mission

CSD Mission Statement:

The expected outcome is a culture where expectations are clear, the path to attain new skills is defined, and metrics are in place to continuously identify development opportunities for our team members.

Case Study: Conagra Brands

40

Examples of Data Trending by Conagra Brands

• Productivity reports

• GMP compliance records

• Downtime reports

• Training completion reports

• Plant scorecards

• Turnover reports

• Staffing reports

• Internal audit reports

• Customer complaint data

• Near miss data

• Organizational assessment

41

CSD Process Design

STEP

04

Planning

Training

Process Review

Validation

STEP

01

STEP

03STEP

02

42

CSD Process Design

STEP

04

Planning

Training

Process Review

Validation

STEP

01

STEP

03STEP

02

Click to edit Master text stylesSecond level

Third level

01 Planning

Process Map

Skills Inventory

Training Curriculum

Qualify Team Members

Set Targets & Metrics

A

B

C

D

E

44

Process Map ExamplePeople Job Role

STEP 1

BlenderGrinderDumper

STEP 2

Stuffing Operators

STEP 3

1 Operator1 Group1 General

Batching

Processing

Total Job Roles 15Total People 41

3 7 3

3 1 3

STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10 STEP 11

7 2 7 2 1 5 1 3

1 1 1 1 1 1 1 1

45

Planning Example

The path to obtain new skills is defined

• Stuffing Operators need 4 major skills for job success

• 7 Stuffing Operators per shift

• Average attendance/staffing is 70%

• Plan to train & validate 10 Stuffing Operators to ensure 7 validated employees are available for that job

– Establishes new tracking metric —review turnover, validated team

• Create training material for those skills using SMEs

– Training content

• Classroom training

• On the job training

– Skills validation checklist

– Establish minimal acceptable scores & remediation actions

46

CSD Process Design

STEP

04

Planning

Training

Process Review

Validation

STEP

01

STEP

03STEP

02

47

02 Training

Agenda

Playbook & OPL

Check for Understanding

On the Job Training

Mentor

A

B

C

D

E

Blending Learning Approach

48

Training and skill expectations are not a secret

• Trainers follow the agendas to complete the classroom training & practical application for the curriculum for Stuffing Operator

• Assign the Stuffing Operator a mentor

• The Skills Inventory coincides with the Classroom, Practical Application and On the Job Training (OJT), and is used as a guide to ensure that the

• Stuffing Operator is instructed in the required skills

• The Skills Validation Checklist is the training tool for the Trainer and the guide for the Stuffing Operator to ensure they learn each task

• The Skills Validation Checklist is also used by the Mentor daily to review each task with the Stuffing Operator

49

Actions of the mentor speak louder than any other step!

Key Role of Mentor

• Mentor vs. Buddy

• Mentor will confirm with Auditor & Supervisor that the trainee is ready for the validation step

• Mentoring is the critical step in the learning process —“what does good look like”

50

CSD Process Design

STEP

04

Planning

Training

Process Review

Validation

STEP

01

STEP

03STEP

02

51

Why Validate?

Ensure learning has occurred

Provide objective metrics for individuals, shifts & the plant

Data to drive action plans Validated team members

03 Validation

Schedule the Audit

Conduct the Audit

Score the Audit

Assess the Audit

Review the Training

A

B

C

D

E

52

Example Validation Process

Auditor Scores

the Audit of

Stuffing Operator

Meet with SupervisorProvide information on the audit

Complete Score

• Deliver scorecard within 30 days• One-on-one private conversations

Incomplete Score

• Develop training action plan• Concerns of trainee

Supervisor Meets with Stuffing Operator

53

54

New Definition of Training Success!

Metrics are in place to continuously identify development opportunities for our team members

“Validated”

Your success will be determined by the number of validated associates, not by the number of people trained

Technical right to win

Step 3 will drive our goal of validation that all associates have and maintain base operating skills

Was training effective?/Did learning occur?

Step 3 validations will provide answers to these basic questions that we often just assume are “yes”

55

CSD Process Design

STEP

04

Planning

Training

Process Review

Validation

STEP

01

STEP

03STEP

02

04 Process Review

Deliver the Scorecard

Review Shift & Plant Results

Monthly Meeting

Targets

Turnover

A

B

C

D

E

56

57

A. Example ScorecardEntityID ItemType EntityTitle CompletionDate GradeCAG12123 VALIDATION-Operations Auditor 5/29/201511:38AMAmerica/NewYork100

CAG6546 VALIDATION-Operations BackLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG6568 VALIDATION-Quality BackLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG6586 VALIDATION-Safety BackLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG361887 OPL BagClipSpringAlignment 8/28/201411:16AMAmerica/NewYork C

CAG7589 VALIDATION ConfinedSpace 6/25/201412:08PMAmerica/NewYork100

CAG16293 VALIDATION Construction&FoodContactSurfaceMaintenance 9/10/201409:19AMAmerica/NewYork100

CAG6541 VALIDATION-Operations CPSValidation 5/29/201511:38AMAmerica/NewYork100

CAG342885 OPL EmergencyEyeWashStations 8/18/201406:13PMAmerica/NewYork C

CAG334886 OPL FirstCaseSticker 7/24/201401:23PMAmerica/NewYork C

CAG6524 VALIDATION-Quality FoodSafety-SQF 5/29/201511:38AMAmerica/NewYork100

CAG6559 VALIDATION-Operations FrontLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG6560 VALIDATION-Quality FrontLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG6564 VALIDATION-Safety FrontLineOperator 5/29/201511:38AMAmerica/NewYork100

CAG17936 VALIDATION Harassment 6/4/201408:42AMAmerica/NewYork 100

CAG6585 VALIDATION-Safety Safety 5/29/201511:38AMAmerica/NewYork100

CAG6539 VALIDATION-Operations TeamLead 5/29/201511:38AMAmerica/NewYork100

CAG6540 VALIDATION-Quality TeamLead 5/29/201511:38AMAmerica/NewYork100

CAG6542 VALIDATION-Safety TeamLead 5/29/201511:38AMAmerica/NewYork100

CAG10833 VALIDATION WorkingAtHeights 6/25/201410:36AMAmerica/NewYork100

Alchemy Course GMPs:PromotingPersonalHygiene(Online)(FoodSafety) 2014-06-1814:50:26 C

Alchemy Course HearingProtection(Online)(WorkplaceSafety) 2014-07-1614:57:47 C

Alchemy Course PreventingFoodContamination(Online)(FoodSafety) 2014-07-2314:54:17 CAlchemy Course ElectricalSafety(Online)(WorkplaceSafety) 2014-08-0607:44:50 C

Alchemy Course EnvironmentalMonitoringBasics(Online)(FoodSafety) 2014-08-1313:57:31 C

Alchemy Course BloodbornePathogens(Online)(WorkplaceSafety) 2014-09-0506:56:22 C

Alchemy Course IntroductiontoFoodAllergens(Online)(FoodSafety) 2014-09-1906:58:01 C

Alchemy Course FirePreventionandProtection(Online)(WorkplaceSafety) 2014-10-1506:58:04 C

Alchemy Course RensCSDShrinkTunnelEquipment&ControlPanelSU(CompanySpecific) 2014-10-1614:35:52 CAlchemy Course IntroductiontoFoodSafetyStandards(Online)(FoodSafety) 2014-10-2913:46:38 C

Alchemy Course Ergonomics(Online)(WorkplaceSafety) 2014-11-0507:31:57 C

Alchemy Course ENTEHSBarricadeTrainingEN2014(CompanySpecific) 2014-11-0507:35:06 C

Alchemy Course BasicFoodFacilityDefense(Online)(FoodSafety) 2014-11-1314:19:23 C

Alchemy Course IntroductiontoHazardCommunication(Online)(WorkplaceSafety) 2015-01-1016:18:51 C

Alchemy Course ENTFSQDocumentsandRecordKeeping1EN2014(CompanySpecific) 2015-01-2306:40:00 CAlchemy Course ENTFSQDocumentsandRecordKeeping2EN2014(CompanySpecific) 2015-01-2306:43:43 C

Alchemy Course RensFS&QDocumentControl2015(CompanySpecific) 2015-01-2306:49:42 C

Alchemy Course CleaningandSanitizinginaDryFoodsFacility(Online)(FoodSafety) 2015-02-2006:52:59 C

Alchemy Course LockOut/TagOutProcedures(Online)(WorkplaceSafety) 2015-02-2506:52:08 C

Alchemy Course PersonalProtectiveEquipment(PPE)(Online)(WorkplaceSafety) 2015-04-0116:16:38 CAlchemy Course DangerousMicroorganismsinFoodManufacturing(Fahrenheit)(Online)(FoodSafety) 2015-04-1215:48:09 C

Alchemy Course OverviewofHACCP(Online)(FoodSafety) 2015-04-3016:17:45 C

Alchemy Course ENTEHSRespiratoryProtectionEN2015(CompanySpecific) 2015-05-1507:09:24 C

Alchemy Course PreventingFoodborneIllness(Online)(FoodSafety) 2015-05-2908:19:05 CAlchemy Course FoodborneIllnessandEmployeeReporting(Online)(FoodSafety) 2015-05-2908:38:20 C

Associate Signature: ___________ Line Lead Signature: ___________ Date: _______

58

Monthly Assessments of Metrics

New Hires – when?Set up a plan

Cross Training Needs

Training for existing employees in new roles

What capital projects are coming up that can be

supported?

What are the audit results for the month?

Did we hand out our scorecards this month on

time?

Is there any material that needs to be changed?

What projects can CSD support to ensure the

changes are sustained?

Do we have validated people trained according to our target numbers?

59

Example: Improvement Highlights

Recordable injuries reduced by

50%

Employee turnover reduced from

23% to 12%Efficiency improved by

32%

Customer complaint reduction of

35%

60

Ideas for Revitalizing Your Training Process

• Define roles and responsibilities

• Request volunteers

• Form teams and focus groups to be engaged in the process

• Kick off Event

• Educate on benefits of process

• Promote a teamwork environment

• Collectively select metrics that is actionable

61

Resources

Alchemy’s Training, Coaching & Reinforcement Programs

62

• Access hundreds of multi-lingual courses on food and workplace safety topics

• Reinforce your training with coordinated huddle guides, digital signage, and posters

• Promote employee-supervisor communication with an award-winning coaching app

• Ensure accurate recordkeeping with automated documentation and audit-ready reporting

63

Q&A

THANK YOUDr. Irene T. BolandChief Learning Officer407.603.7063irene.boland@learningdevelopmentinstitute.com

Laura NelsonVP of Food Safety & Global AlliancesLaura.Nelson@alchemysystems.com