lt j-test construction procedure

TEST CONSTRUCTION PROCEDURES 1

TEST CONSTRUCTION PROCEDURESTEST SPECIFICATION

TEST CONSTRUCTION MODERATION (REVISION)

TRY-OUTTEST ANALYSIS

VALIDATIONPUBLIC AND USER TRIAL

•Baso Jabu•Universitas Negeri Makassar


Test Specifications

A test’s specifications provide the official statement about what the test tests and how it tests it

What is the purpose of the test? What sort of learner will be taking the test?How many sections should the test have?What text types should be chosen –

written/spoken?What language skills should be tested? What language elements should be tested? What sort of tasks are required


Test SpecificationsHow many items are required for each

section? What test methods are to be used?What rubrics are to be used as instructions? Which criteria will be used for assessment by

markers?


Test Specifications Test specifications should include all or some of the following:

The test’s purposeDescription of the test takerTest levelConstruct (theoretical framework for test)Description of suitable language course or textbookNumber of sections/papersTime of each section/paperTarget language situationText-typesText lengthLanguage skills to be tested


Test Specifications Test specifications should include all or some of the following:

Language elements to be testedTest tasksTest methodsRubricsCriteria for markingDescriptions of typical performance of each levelDescriptions of candidates at each level can do in

the real worldSample papers and Samples of students’

performance on tasks


Test Specifications

Three Domains of Taxonomy (Bloom):

CognitiveKnowledge

AffectiveAttitudes

PsychomotorSkills


Test Specifications

CognitiveCognitive domains are objectives which

emphasize remembering or reproducing something which has presumably been learned, as well as objectives which involve the solving of some intellectual tasks for which the individual has to determine the essential problem and then reorder given material or combine it with ideas, methods, or procedures previously learned.


Test SpecificationsCognitive Domain

Six Major Areas

Evaluation

Synthesis

Analysis

Application

Knowledge

Comprehension

New Forms (Verbs)

Creating

Evaluating

Analyzing

Applying

Understanding

Remembering


Test SpecificationCognitive Domain

Knowledge refers to the recall of specific information; Comprehension refers to an understanding of what

was read; Application refers to the converting of abstract

content to concrete situations; Analysis refers to the comparison and contrast of the

content to personal experiences; Synthesis refers to the organization of thoughts, ideas,

and information from the content; and Evaluation refers to the judgment and evaluation of

characters, actions, outcome, etc., for personal reflection and understanding


Test SpecificationCognitive Domain

Remembering indicates recalling information (recognising, listing, describing, retrieving, naming, finding).

Understanding signifies explaining ideas or concepts (interpreting, summarising, paraphrasing, classifying, and explaining).

Applying conveys using information in another familiar situation (implementing, carrying out, using, executing).

Analysing denotes breaking information into parts to explore understandings and relationships (comparing, organising, deconstructing, interrogating, and finding).

Evaluating implies justifying a decision or course of action (checking, hypothesising, critiquing, experimenting, and judging).

Creating refers to generating new ideas, products, or ways of viewing things (designing, constructing, planning, producing, inventing).


Test Specification Affective Domain

Affective domains are objectives, which emphasize a feeling tone, an emotion, or a degree of acceptance or rejection. Affective objectives vary from simple attention to selected phenomena to complex but internally consistent qualities of character and conscience. A large number of such objectives in the literature expressed as interests, attitudes, appreciations, values, and emotional sets or biases.


Test Specification Affective Domain

Affective domains:Spiritual Attitude: pious, virtuousSocial Attitude:

1.Honest to oneself and others2.Disciplined punctual and orderly3.Responsible take risk and apologize4.Tolerance respect others5.Cooperative work together6.Polite respect and behave well7.Self-confident brave and courageous


Test SpecificationPsychomotor Domain

Psychomotor domains are objectives, which emphasize some muscular or motor skill, some manipulation of materials and objectives, or some acts that requires a neuromuscular coordination.


Test SpecificationExample

Subject: _________________________________________Semester/Class: __________________________________School: __________________________________________Test description: _________________________________

NoBasic

Competence

Objective/ Indicators

Domain Techniques Items

1. Listening … C1, C2 MC 1 – 10

2. Speaking C3 … …

3. Reading …4. Writing


Test SpecificationExample: Reading

Subject: _________________________________________Semester/Class: __________________________________School: __________________________________________Test description: _________________________________

Item CognitiveTechnique

s ObjectivesItem Types Remarks

1. C2 MC Finding main idea

Topic sentence

2. C4 MC Comparing Paragraphs

Difference …

3. C1 MC Finding explicit info

Facts or Realities

…

4. … … … … …


Test Construction and ModerationItem Writing and Revision

In writing test items, one should ideally combine both necessary formal professional

The item writing must be based on the test specificationsThe method used for testing a language ability may itself

affect the students’ score, which is called the method effectIt is likely that particular test methods will lend themselves

to testing some abilities, and not be so good at testing othersIn terms of test editing or moderation, each item and the test

as a whole are considered for the degree of match with the test specifications, likely level of difficulty, possible unforeseen problems, ambiguities in the wording of items and of instructions, problems of layout, match between stems and choices, and overall balance of the subtest or paper.


Try-outHowever well designed a test may be, and however

carefully it has been edited, it is not likely to know how it will work until it has been tried out on students.

We do not only need to know how difficult the test items are, but we also need to know whether they work.

The number of students on whom a test should be trialed depends on the importance and type of test, and also the availability of suitable students. The only guiding rule is the more the better, since the more students there are, the less effect change will have on the results.


Test AnalysisThis analysis will show us the extent to which each item works.

For objective test items, traditionally there are two measures of calculation – the facility value and the discrimination index.

For subjectively marked tests although item analysis is inappropriate, such as summaries, essays, and oral interviews, these tests still need to be tried out to see whether the items elicit the intended sample of language; whether the marking system, which should have been drafted during the item writing stage, is usable; and whether the examiners are able to mark consistently.

Once the papers or interviews have been administered, there should be trial marking sessions to see whether the test item prompts have produced the intended kinds of responses, and whether the marking guidelines and criteria are working satisfactorily.


ValidationThe most important question of all in language

testing is validity. The validation process involves the terms internal

and external validity, with the distinction being that internal validity relates to studies of the perceived content of the test and its perceived effect, and external validity relates to studies comparing students’ test scores with measures of their ability gleaned from outside the test.

The commonest types of external validity are concurrent validity and predictive validity.

Expert Validation


Public or User TrialThe tests that have been constructed, tried

out, and analyzed should also be evaluated by public, especially the future users of the tests. The tests are presented to the future users and they analyze the tests, give comments or suggestions to the improvement of the tests, and approve the tests.


Exercises

What are the procedures in constructing a test?

What should be included in the test specifications?

Explain the three domains of taxonomy.What are the new terms in cognitive

domains?

lt j-test construction procedure

Technology