Research to show results!

Wendi Tatum MEd. Kentucky Farm Bureau Mutual Insurance Companies

� What is your company doing for research in the training process?

� How do you evaluate the impacts of a training method? Do you conduct evaluations for all programs?

� Who sees that information? What is it used for?

� How do you know that you have made a positive change in the learners’ behavior or knowledge base?

� Jay needs training for

some Customer Service

Representatives. He

comes to you for training.

You have an idea about

what will help. You want to

be certain that this will

work…

EvaluateTest

HypothesisGather Feedback

and Post-test

Form

Hypothesis Pre-test

Form

Hypothesis

Turn your hypothesis into a question

� Find data sources to collect.

� Identify your sample (control vs. test

group)

Pre-test

�Method 1 : Pre- and Post- tests with

one group of learners.▪ Administer pre-test to the group

�Method 2: Two groups of learners, one

control group and one group with the

training method

▪ Administer pre-test to control group

▪ Administer pre-test to the test group

Test

Hypothesis

Kirkpatrick Review Evaluate

1. Conduct and

document a needs

assessment.

�2. Develop learning

objectives for each

training topic.

4. Develop evaluation

criteria.

�5. Set targets for

evaluation types. Some

examples: 100% L1, 80%

L2, 25% L3, 5-10% L4

�7. Track the data

Conduct Cost/ Benefit ROI

8. Communicate the

process and

incorporate into the

management system.

Steps to create an

effective

evaluation process

� Action Research

� Does _______ solve the problem or

validate a theory (educated guess) or

hypothesis?

� T-test

� The t-test is probably the most commonly

used procedure for hypothesis testing.

� Actually, there are several kinds of t-tests, but

the most common is the "two-sample t-test“ or

“students t-test” or the "independent samples t-

test".

� http://web.mst.edu/~psyworld/texample.ht

m

� http://ccnmtl.columbia.edu/projects/qmss/t

he_ttest/twosample_ttest.html

� http://www.markettruths.com/process/exa

mples/ttest.asp

� Problem: Freshman students use of grammar and writing conventions.

� Rationale: 2003-2004 ISTEP+ results below average in writing.

� Plan: Integrate rubrics into the ninth grade OLC curriculum.

� Question: Will rubrics improve students post-test scores and correct poor grammar?

� Data Collection: Writing Samples, Rubric Assignments, pre- and post- test data (4 week period)

http://www.graphpad.com/quickcalcs/ttest1.cfm

� Isolate one item.

� Anticipate and plan for

glitches. They will

happen.

� Set targets for your

evaluations .

� Track your data.

� Communicate the

process and incorporate

into the management

system.

EvaluateGather Feedback

and Post-test

Form

Hypothesis

Test

Hypothesis

Pre-test

Empirical Analysis for Training

Think about a problem

related to technology or training that you are

trying to solve at your

company right now..

List your problem

______________

Identify the ProblemIdentify the ProblemIdentify the ProblemIdentify the Problem

Learning Experience

Evaluation

Empirical Research and

Training

Did the presentation meet your

expectations?

1. Yes, it exceeded my

expectations

2. No, it didn’t meet my

expectations

Did you learn new information

in this session?

1. Yes

2. No

I will go back and implement pre-

and post- tests.

Yes No

I am

alre

ady

doin

g th

is

100%

0%0%

1. Yes

2. No

3. I am already doing this

I will go back and implement a t-

test with my pre- and post- tests.

1. Yes

2. No

3. I am already doing this

The length of presentation

was..?

1. just right

2. too long

3. too short

The presenter was

knowledgeable, well prepared

and presented material clearly

and effectively?

1. Strongly agree

2. Agree

3. Disagree

4. Strongly Disagree

Please feel free to leave any other

comments on the card in front of

you.

Contact: