testing and evaluation copyright © 2013 georgia public safety training center

TESTING AND EVALUATION

COPYRIGHT © 2013 GEORGIA PUBLIC SAFETY TRAINING CENTER www.gpstc.org

2

Prepare for a test!

This is the “squares test”

3

How Many Squares Do You See?

Answer?

4

How Many Squares Do You See? (contd)

Answer?

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

5

Answer?2019

1817

How Many Squares Do You See? (contd)

6

Answer?

22

21

How Many Squares Do You See? (contd)

7

Answer?

2423

How Many Squares Do You See? (contd)

8

Answer?

25

How Many Squares Do You See? (contd)

9

Answer?

26

How Many Squares Do You See? (contd)

10

Answer?

27

How Many Squares Do You See? (contd)

11

Answer?

28

How Many Squares Do You See? (contd)

12

Answer?

29

How Many Squares Do You See? (contd)

13

Answer?

30

How Many Squares Do You See? (contd)

14

Terminal Performance Objective

For a chosen topic, the public safety training instructor will construct a variety of different types of written exam questions, in accordance with the American Society for Training and Development (ASTD) Learning System.

15

Enabling Objectives

• Define the term Evaluation.

• Explain how the instructor’s performance can be improved through evaluations.

• Explain the purpose of testing.

• Discuss the concept of performance testing.

16

Enabling Objectives (contd)• List the steps in administering and grading

written tests.

• List the characteristics of a good test.

• Compare and contrast the different types of written test questions.

• Construct written test questions for a total of five (5), three (3) different types.

17

To Test & Evaluate

• A jury WEIGHS evidence.• A teacher GRADES students.• An antique dealer APPRAISES an article.• A panel of experts JUDGES a beauty contest.• A realtor ASSESSES the value of real estate.

18

Accountability

A TWO WAY STREET• Students have a

responsibility to LEARN

• Instructors have an obligation to TEACH.

19

• Provides an estimate of achievement for some useful purpose.

• Appraising the adequacy of student performance.

• Determines how well both the instructor and student are progressing toward specific objectives.

• Continuous process of observing and measuring changes in behavior.

What is Evaluation?

20

Evaluation Triangle

JUDGMENT

CRITERIA EVIDENCE

21

Evaluating Your Own Performance

• Pre-test and Post-test

• Indicator of your performance

• Feedback for possible Lesson Plan revision

22

Purpose of Testing

• Why Test?– Improve Teaching/Learning Process– Aid in Instruction

• Gaps in Learning• Gaps in Instruction

– Reinforce Learning• Stimulate Thought• Encourages Learning• Improves Study Habits

23

Purpose of Testing (contd)

• Tests Provides– Incentives

• To Study• Sense of Competition

• Assign Grades– Valid– Comprehensive– Differentiating

Performance Testing

What is the best way to determine if students can perform a specific task?

Have them do it, or at least simulate the activity.

Require actions such as identification, simulation and work samples.

25

Performance Testing (contd)

• Psychomotor Skills– Measure specific skills that require action

• Report writing

• Handling tools

• Weapons

• CPR

• Approach to emergency scenes

26

Performance Testing (contd)

• Psychomotor Skills– Practical Exercise

• Practice skills and get feedback and coaching

– Performance Exam• Demonstrate proficiency under controlled

conditions

27

Performance Testing (contd)

• Advantages– Face Validity

– Job Related

– Reliable

– Observe Individual Differences

• Disadvantages– Not Comprehensive– Time Consuming– Tools, Equipment,

Material– Design Difficult– May be Unreliable

28

Performance Testing (contd)

• Construction and Use– Step One: Specify Performance

Objective to be measured– Step Two: Select Rating Factors (Use a

Form)– Step Three: Prepare directions– Step Four: Try out if new– Step Five: Use more than one rater– Step Six: Follow procedures– Step Seven: Give immediate feedback– Step Eight: Rotate teams and

members

29

Written Tests

Often limited to lower cognitive learning levels of recall and recognition

No set “correct” response. Can vary with each student’s solution to the problem.

OBJECTIVE

SUBJECTIVE

30

Written Tests (contd)

• Classified According to Purpose– Prescriptive

• Measuring readiness• Determining placement

– Progress• Measure improvement• Diagnose learning difficulty

– Comprehensive• Measure terminal performance

31

Written Tests (contd)

• Delivery and Use of Test– Instructions– Security– Administration

• Before: notify, tell when, what, how and what they should bring

• During: good environmental conditions, relieve stress, encourage to do own work

• After: give results, review and evaluate test

32

Characteristics of a Good Test• Measures the material taught

• Yields consistent results

• Scorer bias eliminated

• Liberal sampling of course materials

• Detects small differences in student achievement

• Effective/efficient administration

VALID

USABLE

OBJECTIVE

COMPREHENSIVE

DIFFERENTIATING

RELIABLE

33

Essay (Subjective)

• Advantages– Relatively easy to

construct

– Effectiveness in writing

– Offers opportunity to select, organize, and integrate facts

• Disadvantages– Low validity

– Reliability low

– Time consuming

34

Essay Example

Question: Explain the difference between hearing and listening. Include any key points or limitations of either process.

35

True/False (Objective)

• Disadvantages– 50/50

– Does not reflect how much they really know

– Hard to write

• Advantages– Test recall and

recognition

36

True/False Example

Question: You should always anticipate danger.

A) TrueB) False

37

Fill-in-the-blank (Objective or Subjective)

• Advantages– Demonstrates

knowledge

• Disadvantages– Hard to construct

38

Fill-in-the-blank (contd) Example

Question: You have a better chance of making the right decision if

you ______ before you act.

39

Listing (Objective or Subjective)

• Advantages– Demonstrate

knowledge

• Disadvantages– Subjectivity

– Grading

40

Listing Example

Question: List the three different heat-related medical emergencies in order from least severe to most severe.

41

Multiple Choice (Objective)

• Advantages– Test recall and

recognition

• Disadvantages– Hard to design

– Lengthy preparation time

42

Writing Multiple Choice Items

• Terminology– Stem

– Options

– Key

– Distractors

43

Multiple Choice Example

Question: The human heart has __ chambers.

a. 1b. 2c. 3d. 4

44

Pointers for Multiple Choice

• No questions answering each other• Logical answer• Avoid “none of the above” and “all of the

above”• Same length answers• Answers should be in ascending order• Only 1 correct answer with distractors• Highlight keywords: IS, NOT, EXCEPT

45

Matching (Objective)

• Advantages– Test recall and

recognition

• Disadvantages– Hard to construct

(distracters)

Matching Example

Question: Given a household fire extinguisher, match the picture symbol to the class fuel source it will mitigate.

B

A

C

47

Questions in General

• Minimize length of questions

• Questions do not answer themselves

• Answers to question not obvious

• No trick questions

• Questions MUST relate to your PERFORMANCE OBJECTIVES

48

Enabling Objectives• Define the term Evaluation.• Explain how the instructor’s performance can be

improved through evaluations.• Explain the purpose of testing.• Discuss the concept of performance testing.• List the steps in administering and grading written

tests.• List the characteristics of a good test.• Compare and contrast the different types of written

test questions.• Construct written test questions for a total of five

(5), three (3) different types.