test construction and evaluation final

7/31/2019 Test Construction and Evaluation Final

1/12

1

Language testing at any level is a highly complex undertaking that must be based on

theory as well as practice. The guiding principles that govern good test design,

development and analysis are validity, reliability, practicality and washback.

1.1. ValidityThe term validity refers to the extent to which a test measures what it says it measures. In other

words, test what you teach, how you teach it. Types of validity include:

Content validitymeans that the test assesses the course content and outcomes using formatsfamiliar to the students. Construct validity refers to the fit between the underlying theories and methodology oflanguage learning and the type of assessment. Face validity means that the test looks as though it measures what it is supposed tomeasure. This is an important factor for both students and administrators. Other types of

validity are more appropriate to large-scale assessment.

1.2. ReliabilityReliability refers to the consistency of test scores. It simply means that a test would give similar

results if it were given at another time. Three important factors affect test reliability.

Test factors such as the formats and content of the questions and the length of the exam mustbe consistent. Administrative factors are also important for reliability. These include the classroom setting(lighting, seating arrangements, acoustics, lack of intrusive noise etc.) and how the teacher

manages the exam administration. Affectivefactors in the response of individual students can also affect reliability. Testanxiety can be alleviated by coaching students in good test-taking strategies.

1.3. Practicality


2/12

2

Classroom teachers are well familiar with practical issues, but they need to think of how practical

matters relate to testing. A good classroom test should be teacher-friendly. A teacher should be

able to develop, administer and mark it within the available time and with available resources.

Good design consists of a number of stages which progressively elaborate the design. Good test

design consists of a number of stages which progressively elaborate the design of tests:

2.1. Construct Specifications:What theoretical model is going to be tested? (skills / structures /functions /vocabulary)

2.2. Format Selection:What types of exercises are available for testing the model? (close-ended tests / open-ended

response tests / restricted-response tests)

2.3. Item Writing:To what extent are the instructions clear and precisely focused?


3/12

3

4.1.1. Reading performance standards

Test designers should take into account the five standards areas as stated in the Guidelines (2007).

Students have to demonstrate their level of attainment through performing reading tasks as specified

in the syllabus and Guidelines:

Skim a text for general idea/main idea; Scan a text for specific information; Make inferences: informational, explanatory, propositional and pragmatic; Infer word meaning from the context; Identify discourse markers and their functions in the text; Distinguish main idea from detail; Distinguish fact from opinion; Distinguish cause from effect; Identify the authors attitude/intention Transfer information from text to chart, diagram or table. Re-arrange the ideas or topics discussed in the text. Evaluate ideas expressed in a text (i.e. critical reading). Draw conclusions from the text.

4.1.2. Text types and topics

Reading texts could be in the form of newspaper articles, letters, biographies, encyclopedia entries,

film/book reviews, report and advertisement. They can be narrative, informative, descriptive, or

argumentative texts. Texts designed for testing reading need to be of appropriate level of difficulty.


4/12

4

4.1.3. Text length

Streams Number of words

Arts 400 - 450

Humanities 350 - 400

All science, technical and Original streams 250 - 300

4.1.4. Test techniques

The following techniques should allow testers to evaluate testees various reading competencies.

Test techniques should vary so that no particular test technique would become predictable.

Wh/questions True/False + appropriate justification Sentence completion Summary cloze Gap filling Information transfer Matching elements Extracting information from the text Ordering elements of the text (events, arguments, etc.)

Test techniques should lead to clear and reliable scoring for the sake of equity. Three to six of the

above techniques must appear in the reading comprehension section to guarantee appropriate

coverage of the reading performance standards.

4.1.5. RubricsRubrics are those characteristics of the test that guide the testees and inform them about the

procedures for performing test tasks. They also inform testees about how the test will be scored. Forthe sake of standardization, the following rubrics should appear in the test paper:

Which of the following would be the best title for the passage? Which of the following could be inferred from the text? Which of the following questions does the passage answer? Are these sentences true or false? Justify Answer these questions in your own words Complete these sentences in your own words Fill in the blanks with a word from the text Fill in each blank with an expression from the text Find in the text words or expressions that mean the same as.. What do the underlined words in the text refer to? Pick out from the text expressions or phrases which show that Match the main idea with the corresponding paragraph In what order do the following ideas/arguments/events appear in the text? What is the writers opinion/attitude/tone/intent? Choose the right answer from the list given Fill in the chart with the right information from the text


5/12

5

It comprises three content areas: vocabulary, grammar and rhetorical functions. It aims at assessing

students ability to use language accurately and appropriately.

4.2.1. Vocabulary

Vocabulary test items should be related to the topics and themes dealt with in the syllabus and

textbook(s). It should particularly focus on the following areas:

collocations Word associations word definitions synonyms/ antonyms poly-words word formation suffixes, prefixes and their meanings phrasal verbs

idiomatic expressions.

4.2.2. Grammar

Grammar test items assess students ability to recognize, understand and apply rules and

their grammatical forms. They also evaluate their competence in using those forms accurately

and appropriately. The test items should be realistic and contextualised. The grammar test

component must include items related to the following list of structures:

Tenses (including past perfect, past perfect continuous and future perfect); Wishes(present and past), If only (past), and conditionals (including type 3); Reporting commands, statements, and questions in the present and past; Modals: present and past; Infinitive vs. Gerund; Phrasal verbs with particular focus on the meaning of particles ; Passive with different tenses; Passive with modals; Restrictive and non-restrictive clauses; Linking words.

4.2.3. Rethorical Functions

The rhetorical functions to be tested at this level are the following:

Defining Cause and effect Purpose Addition Concession.

4.2.4. Language Test Techniques


6/12

6

A variety of test techniques should be included in the test paper. The overuse of some of these

techniques at the expense of others must be avoided.

Correct form Sentence transformation/ rewriting Sentence completion Gap filling Matching Multiple choice Cloze procedure with list or with first letter given Word definitions Classifying/categorising Providing synonyms, antonyms or poly-words.

4.2.5. Rubrics

The rubrics below are to be used when testing grammar, vocabulary and rhetorical functions.

PUT THE VERBS BETWEEN BRACKETS IN THE CORRECT TENSE. FILL IN THE GAP(S) WITH THE APPROPRIATE WORD OR EXPRESSION FROM THE LIST. FILL IN THE GAP WITH AN APPROPRIATE WORD OR EXPRESSION. FILL IN THE GAP WITH AN APPROPRIATE PHRASAL VERB. FILL IN THE GAP WITH THE APPROPRIATE COLLOCATION. MATCH THE WORDS THAT GO TOGETHER TO MAKE APPROPRIATE COLLOCATIONS. REWRITE THE SENTENCES BEGINNING WITH THE WORDS GIVEN. REWRITE THE SENTENCES AS INDICATED. MATCH THE WORDS WITH THE CORRESPONDING DEFINITIONS/ SYNONYMS/ ANTONYMS. REPLACE THE UNDERLINED WORD WITH THE APPROPRIATE SYNONYM. SELECT THE ONE WORD/PHRASE THAT CLOSELY MATCHES THE MEANING OF THE

UNDERLINED WORD.

PROVIDE THE APPROPRIATE WORD(S) FOR THE FOLLOWING DEFINITION.

GIVE THE CORRECT FORM OF THE WORDS BETWEEN BRACKETS. JOIN THE PAIRS OF SENTENCES WITH THE LINKING WORDS GIVEN. MATCH EACH SENTENCE WITH ITS APPRORIATE FUNCTION.


7/12

7

4.3.1. Writing performance standardsThe writing section assesses students writing skill and provides an indication of how well learners

communicate in the written language. Testers can determine learners achievement in writing

through real-life tasks, bearing in mind the performance standards stated in the syllabus and

guidelines:

using adequate and relevant content; using an appropriate text structure according to the conventions of the targeted genre ; selecting an appropriate style according to audience; using cohesive devices and transitions to organize content at the level of paragraph(s) and

whole text effectively;

using mechanics (spelling, punctuation, capitalization) and applying the appropriatestructures and vocabulary to get the message across;

using a variety of sentence structures: simple, complex sentences, active and passive forms,parallelism, etc.;

The followinggenres

are highly recommended in the testing of writing:

formal/ informal letters, film review, book review, narrative, descriptive, expository text, and argumentative texts.

Topics designed for testing writing should be bias-free and should target stimulating issues related

to the themes familiar to the students and/or dealt with in the textbooks. Writing tasks should be

well defined; i.e. students should know exactly what they are supposed to write. The audience and

purpose should also be specified clearly in the task. Additional useful notes could be provided with

the writing prompts to help students demonstrate their competence to the fullest.

4.3.2. The writing length

The writing task must specify the number of words as indicated in the table below:

Arts Humanities All science, technical and original streams

approximately 250

words

approximately 200

words

approximately 150 words


8/12

8

--------------------------------------------------------------------------------------------------------------------------------

I was walking around on a Saturday afternoon. It was cold and it was starting to rain. I was looking for a

restaurant or a caf where I could sit and have something to drink. When I saw one, I crossed the street and I

was there. I sat at a round table and asked for a cup of coffee. While I was waiting for my order, I realised

that there were other people in the place, but I sensed loneliness. I saw their bodies, but I couldn't feel their

souls. That was because their souls didn't belong to them; instead they belonged to the Internet.

I stood up and walked between the tables. When I came to the biggest computer, Isaw a thin, small man sitting in front of it. "I'm Steve", he finally answered after I

had asked him a couple of times what his name was. "I can't talk with you. I'm

busy," he said. I thought he was working, and I apologised. He didn't say anything.

Before going back to my seat, I wanted to see what he was doing. I stood behind

him. He was not working. He was chatting online with somebody - probably

someone he didn't know--and, at the same time, he was playing a computer game -

a war game. I was astonished.

Why didn't Steve want to talk to me? I tried again to communicate with that computer geek, but not a word

came out of his mouth. I touched his shoulder, but there was no reaction. I was getting upset. I put my hand

in front of the monitor, and he started shouting, "leave me alone!" I took a few steps back. I was ashamed. I

wanted to look and see if all those people in the caf were looking at me, so I raised my head, but there wasnobody interested in what was happening. I walked back to my table, and I noticed everybody was using a

computer.

At that moment, I realised that the people in that place were having a cup of coffee and a nice conversation

with their machines, not with people. All of them were more interested in having a relationship with the

computer, particularly Steve. I wouldn't want to imagine the future of human beings if they preferred sharing

their lives with machines instead of people. I was worried and I sank in my thoughts. I didn't even realise that

the coffee was bad, just as Steve didn't even realise that there was a person next to him.

I. COMPREHENSION (15 POINTS)BASE YOUR ANSWERS ON THE TEXTA. CHOOSE THE BEST TITLE FOR THE PASSAGE. TICK THE APPROPRIATE BOX (2 pts )Computer games

Computer programming

Computer addiction

B. ARE THESE SENTENCES TRUE OR FALSE? JUSTIFY. ( 3 pts )1. The writer went to a caf on a sunny day. .

. 2. Steve answered the writers question immediately. .

. 3. The people in the caf didnt care about what was going on between Steve and the writer. ..

C. ANSWER THESE QUESTIONS. (3 pts)

1. Why did the writer apologise? .....

2. What was Steve busy doing? .. 3. Why did Steve start shouting? ....

.

D. COMPLETE THESE SENTENCES (3 pts)1. The writer felt lonely in the caf although...

2. When the writer saw that Steve was too busy, he wanted to ......3. The writer cant imagine a future where people .


9/12

9

E. WHAT DO THE UNDERLINED WORDS IN THE TEXT REFER TO? (2 pts)1. they: ..2. machines: .

F. FIND WORDS IN THE TEXT THAT MEAN THE SAME AS: (2 pts )1. surprised (paragraph 2): .

2. computer screen (paragraph 3): .

II. LANGUAGE (15 POINTS)A. FILL IN THE GAPS WITH THE CORRECT PHRASAL VERBS. (2 pts)

set upcome backcalm downlook upfill inapply forI need to .. this application form. I want to .. a visa to Canada. Im going to study

management there and hope to .a new business when I to Morocco.

B. PUT THE VERBS IN BRACKETS IN THE CORRECT TENSE OR FORM. (3 pts)

Bob and David enjoy (watch) ...science fiction films. Yesterday, they (go)

to the cinema, but when they arrived, they found that the film (already/start)

. So, they decided to have a walk along the coast.

C. JOIN THE PAIRS OF SENTENCES WITH THE WORDS BETWEEN BRACKETS. (2 pts)

1. Jamal spent all the afternoon chatting. He didnt have time to do his homework. (because)

2. My friends decided to go on an excursion to the Atlas Mountains. They might face difficulties.

(despite)

D. REWRITE THE SENTENCES BEGINNING WITH THE WORDS GIVEN. (4pts)

1. What a pity! I didnt send them the message on t ime.

If only

2. Why dont you eat more vegetables and fruit? the doctor said.

The doctor advised the patient..

3. You must send your application before the end of March.

Your application ..

4. I dont have a modem, so I cant have access to the Internet.

If

E. GIVE THE CORRECT FORM OF THE WORDS BETWEEN BRACKETS. (2 pts; 0.5 pt each)

Active (citizen) .......................... can be defined as the (involve) ....................... of citizens in public

life through their (participate) ...................................... in the communities' projects that can

(contribution).. to the improvement of people's living conditions.

F. MATCH THE EXPRESSIONS WITH THEIR APPROPRIATE FUNCTIONS. (2 pts)1. Youd better do some exercises in the workbook.

2. Could you close the window, please?a. requesting

b. apologising

c. suggesting

III.WRITING (10 POINTS)Write an email to your friend describing the best place you have ever visited .

(Approximately 250 words)

Here are some cues to help you:


10/12

10

-Name of the place (city, village, countryside, forest, beach, ..)- Time of the visit (weekend, Sunday, holidays, summer , )- Characteristics (attractive, nice, interesting, modern, traditional, . )- Your feelings (happy, satisfied, unforgettable, memorable, . )

..


11/12

11

Consider the appropriateness of the text (for the target level) in the light of the

following guidelines:

1. Is the text at an appropriate level for the students (language/concepts)?2.

Is the text culturally biased? Does its cultural load impair comprehension?

3. Does the sampling of the text suit syllabus topics?4. Is the text similar to the reading material students are familiar with?5. Does the text lend itself to a sufficient number of comprehension tasks/questions?6. Does the text allow the assessment of high order comprehension skills/subskills?7. Is the text well organized, logically sound and factually correct?8. Does the text require outside information for its full comprehension?9. Is the text taken for a known/familiar source?10.Is the text too long/too short?

Consider the appropriateness of the comprehension questions in the light of the

following guidelines:

1. Are the objectives behind the questions clear (specification of skills/subskills to assess) ?2. Can any questions be answered without recourse to the reading passage?3. Do the questions test significant material in the text?4. Are the questions written in simple language?5. Are there questions on every part of the text?6. Are the questions worded in a language that is similar to that of the text?7. Do any questions give away the answers to other questions?8. Are the questions independent?9. Are the questions varied in type?10.Are any questions overlapping?11.Do any questions have more than one possible answer?12.Are any questions centred on the same information in the text?

Consider the appropriateness of the language questions in the light of the following

guidelines:


12/12

12

1. Do the items assess key elements in the syllabus?2. Are the stems short/simple/clear and with enough context?3. Is the language of the stem natural?4. Is the grammatical structure of the stem familiar to the students?5. Is the item embedded in a context?6. Is the context natural and meaningful?7. Are the content and reliability level of the items suitable for the average student?8. Are there many possible answers to any items?9. Is there any mismatch between the way items are tested and the way they have been taught?

Consider the appropriateness of the writing task in the light of the following

guidelines:

1. Is the wording of the topic clear and in simple/familiar language to the student?2. Is the writing task authentic?3. Does the task specify the audience/ purpose and text type?4. Is the input concise and with enough information for the accomplishment of the task?5. Do students know exactly what is expected of them in terms of content/size?6. Is the task based on a topic which members of a sex may be favored?7. Work out a scoring scheme based on the writing task to be performed

test construction and evaluation final

Documents