BNHS-Villa Maria AnnexIn-Service Seminar

Oct. 21-24, 2014

WELCOME

TEST

CONSTRUCTION

Presented by:

Arnel O. Rivera

http://www.slideshare.net/ArnelSSI

The Art of Effective Evaluation

Presented to the Faculty of BNHS-Villa Maria Annex

Coping with the K-12 Dilemma

EMPTY YOUR CUP

An Old Buddhist Tale

Bruce Lee, the legendary

martial artist, once told a

story about a highly-

educated man who went

to a Zen teacher to ask

and acquire knowledge

about Zen.

As the Zen teacher began

to explain things, the man

would frequently interrupt

with remarks like, "Oh,

yes, we have that too…”

Finally the Zen teacher

stopped talking. He began

to serve tea. He poured

tea into the man’s cup.

He poured tea into the cup

until it was full, and then

kept pouring until the cup

overflowed. "Enough!" the

man once more

interrupted, "No more can

go into the cup!"

“Indeed, I see," answered the Zen teacher, "If

you do not first empty the cup, how can you

taste my cup of tea?"

A full cup cannot accept

anything more. Similarly,

a person who believes

that he had learnt a lot

cannot learn anything

else & will stagnate

quickly and not move to

higher levels.

ATTITUDE DETERMINES YOUR ALTITUDE.

A SMALL CHANGE IN ATTITUDE MAKES BIG

DIFFERENCE

HOUSE RULES

• Put you cellphones in silent mode. No

texting of entertaining of calls.

• No multi-tasking. Concentrate on the

seminar.

• BAWAL ANG DEDMA!

Participate, Participate

and PARTICIPATE!

Why is there a need to study TEST

CONSTRUCTION?

For every problem there is a solution. If you are not part of the solution then you are part of the PROBLEM.

Chinkee Tan

Chink Positive

13% of students who got low grades in exams are caused by faulty test questions.

WORLDWATCH

The Philadelphia Trumpet

August 2005

Sample Test Question:

“The evaluation of pupils’

progress is a major aspect of

the teacher’s job.“Evaluating Educational Outcomes

(Oriondo & Antonio)

Explain the message of the comic strip.

Our students were trained to memorize information.

After they have memorized facts, then what?

Old belief about teaching .....

Education is a process where the notes of the teachers or the contents in the books are transfered to the notebooks of the students, without understanding them.

(Wiggins, 2009)

In your lesson plans, did you ever bother to ask this question:

• How do you know if you have achieved your desired result?

“A test is a sample of behavior, products, answers, or

performances from a particular domain” (Carrington, 1994)

“… it's a systematic method of eliciting performance which is intended

to be the basis for some sort of decision making" (Hughes, 1989).

“A test will predict performance levels, and the learner will somehow

reconstruct its parts in meaningful situations when necessary”

(McCann, 2000)

“ Testing is generally concerned with turning performance into

numbers.” (Baxter, 1997)

Guidelines for Test Construction

What is testing?

Inform learners and teachers of the strengths and

Motivate learners to review or consolidate specific material

Guide the planning/development of the ongoing teaching

Determine if the objectives

Encourage improvement

Create a sense of accomplishment

Guidelines for Test Construction

What are tests for?

weaknesses of the process

process

have been achieved

Encourage

improvement!

BARRIERS IN TEST CONSTRUCTION

Ms. Alanganin –

Mr. Highfalutin –

Ms. Madaldal –

Ms. Magulo –

Ms. Malabo –

Mr. Pulpol –

Ms. Foringer –

Ms. Colonial Mentality –

confusing statements

difficult vocabulary

excessive wordiness

complex sentence structure

unclear instructions

unclear illustrative materials

linguistically bound words

culturally bound words

Characteristics of Good Tests

Validity – the extent to which the

test measures what it intends to

measure

Reliability – the consistency with

which a test measures what it is

supposed to measure

Usability – the test can be

administered with ease, clarity

and uniformity

Scorability – easy to score

Interpretability – test results can

be properly interpreted and is a

major basis in making sound

educational decisions

Economical – the test can be

reused without compromising the

validity and reliability

Other Things to Consider

“To be able to prepare a good

test, one has to have a

mastery of the subject

matter, knowledge of the

pupils to be tested, skill in

verbal expression and the

use of the different test

format”

Evaluating Educational Outcomes

(Oriondo & Antonio)

1. Multiple Choice

2. True or False

3. Matching Type

4. Fill-in the blanks (Sentence Completion)

5. Essay

5 Most Commonly used

Test Format

Source: Turn-out of Test Questions in SSI (2003-2007)

General Steps in Test Construction

DRAFT

ORDER

TEST ANALYZE

SUBMISSION

PRODUCE A

T.O.S.

Table of Specifications (TOS)

A two way chart that relates the learning outcomes to the course content

It enables the teacher to prepare a test containing a representative sample of student behavior in each of the areas tested.

Sample TOS

KPUP Paradox

Knowledge – answers the question what, where and when (ano, saan at kailan)

Process – answers the question how? (paano)

Understanding – answers the question why (bakit)

Tips in Preparing the Table of Specifications (TOS)

Don’t make it overly detailed.

It's best to identify major ideas and skills

rather than specific details.

Use a cognitive taxonomy that is most

appropriate to your discipline.

Weigh the appropriateness of the distribution

of checks against the students' level, the

importance of the test, the amount of time

available.

General Rules in Writing Test

Questions

Number test questions continuously.

Keep your test question in each test group uniform.

Make your layout presentable.

Do not put too many test questions in one test group. T or F: 10 – 15 questions

Multiple Choice: max. of 30 questions

Matching type: 5 questions per test group

Others: 5 – 10 questions

Some additional guidelines to consider when writing items are described below:

1. Avoid humorous items. Classroom testing is

very important and humorous items may

cause students to either not take the exam

seriously, become confused or anxious.

2. Items should measure one’s knowledge of the

item context not their level of interest.

3. Write items to measure what students know,

not what they do not know. (Cohen & Wallack)

Multiple Choice Test

When checking the stems for correctness:

Ensure that the stem asks a clear

question.

Reading level is appropriate to the

students

The stem is grammatically correct.

Negatively stated stems are

discouraged.

What to Look for on

Multiple Choice Tests

Example:

What is the effect of releasing a ball in

positive gravity?

a) It will fall “down.” correct

b) It will retain its mass. true but unrelated

c) It will rise. false but related

d) Its shape will change. false and unrelated

Anatomy of a Perfect

Multiple Choice Tests

Multiple Choice Questions

1. Use negatively stated stems sparingly and

when using negatives such as NOT,

underline or bold the print.

2. Use none of the above and all of the above

sparingly, and when you do use them, don't

always make them the right answer.

3. Only one option should be correct or clearly

best.

Multiple Choice Questions:

4. All options should be homogenous and

nearly equal in length.

5. The stem (question) should contain only

one main idea.

6. Keep all options either singular or plural.

7. Have four or five responses per stem

(question).

Multiple Choice Questions:

7. When using incomplete statements place the blank space at the end.

8. When possible organize the responses.

9. Reduce wordiness.

10. When writing distracters, think of incorrect responses that students might make.

Examples

1. Sheldon developed a highly controversial theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work?

a. He was influenced too much by the Freudian psychoanalysis.

b. His rating of physique and temperament were not independent.

c. He failed to use empirical approach.d. His research sample was improperly

selected.

Examples

Better: (Eliminate excessive wording and irrelevant information)

1. Which of the following is a criticism of Sheldon's theory of personality?

Examples

1. The receptors for the vestibular sense are located

a. in the fovea.b. in the brain.c. in the middle ear.d. in the inner ear.

Examples

Better: (Include in the stem any word(s) that might otherwise be repeated in each option.)

1. The receptors for the vestibular senses are located in the _______.

a. foveab. brainc. middle eard. inner ear

True or False

Each statement is clearly true or clearly false.

Trivial details should not make a statement false.

Statements are written concisely without more elaboration than necessary.

Statements are NOT quoted exactly from text.

What to Look for on

True/False Tests

Give emphasis on the use of quantitative terms than qualitative terms.Avoid using of specific determiners which usually gives a clue to the answer. False = all, always, never, every, none,

only True = generally, sometimes, usually,

maybe, often

Discourage the use of negative statements.Whenever a controversial statement is used, the authority should be quoted.Discourage the use of pattern for answers.

Tips in Making True/False Tests

Examples:

____ 1. Repetition always strengthens the tendency for a response to occur.

(Using "always" usually means the answer is false.)

Find the errors, and/or problems with the following true-false tests.

Examples:

_____ 2. The process of extinction is seldom immediate but extends over a number of trials.

(Words like "seldom" usually indicate a true statement.)

Examples:

_____ 3. The mean, median, and mode are measures of central tendency, whereas the standard deviation and range are measures of variability.

(Express a single idea in each statement.) e.g.“The mean and standard deviation are measures of central tendency.”

Matching Type

Parts of the Matching Type Test

(Vertical Type)

Column A

(Premise)

Column B

(Response)

Parts of the Matching Type Test

(Horizontal Type)

(Response)

(Premise)

The list of responses should be relatively short.

Response options should be arranged alphabetically or numerically.

Directions clearly indicate the basis for matching.

Can responses be used more than once?

Where will you place your answer?

Can students infer relationships or are they based on real word logic?

What to Look for on

Matching Type Tests

Position of matches should be varied.

Avoid using patterns.

The choices of each matching set

should be on one page

There are more responses than

premises in a single set if responses

cannot be used more than once.

What to Look for on

Matching Type Tests

The premises are homogeneous as well as the responses and are grouped as one item.

Example: Set A: Provinces in Region I

Set B: Provinces in CAR

If responses can be used more than once, it should be proportional to the number of premises (3:5 or 4:10)

What to Look for on

Matching Type Tests

Examples:

Directions: Match the following.

1. Food A. Primary reinforcer

2. Psychoanalysis B. Sigmund Freud

3. B.F. Skinner C. Operant conditioning

4. Standard deviation D. Measure of variability

5. Schizophrenia E. Hallucinations

Examples:Better: (Use homogenous material in matching items, and if responses are not to be used more than once, include more responses than stimuli.)

Match the theories in Column A with their proponents in Column B. Write the letter of the correct answer.

Column A Column B

___ 1. Psychodynamic Theory A. Albert Bandura

___ 2. Trait Theory B. B.F. Skinner

___ 3. Behaviorism C. Carl Rogers

___ 4. Humanism D. Gordon Allport

___ 5. Social Learning Theory E. Karn Horney

F. Raymond Cattell

G. Sigmund Freud

Proposed Arrangement of Test Items

True or False

Multiple Choice

Matching Type

Sentence Completion

Others (RRT/Analogy/CST)

Essay

Things to Remember:Making a good test takes time

Teachers have the obligation to provide their students with the best evaluation

Tests play an essential role in the life of the students, parents, teachers and other educators

Break any of the rules when you have a good reason for doing so! (emphasis mine)

(Mehrens, 1973)

POINTS TO PONDER…

A good lesson makes a good question

A good question makes a good content

A good content makes a good test

A good test makes a good grade

A good grade makes a good student

A good student makes a good COMMUNITY

Jesus Ochave Ph.D.

VP Research Planning & Development

Philippine Normal University

For questions , comments or if you want to

download this file, log-on to:

http://www.slideshare.net/ArnelSSI