LOGO
Dinh Cong Triet
TEST DESIGNING &
TEST SPECIFICATION BUILDING
Summer training course- August 2011
10 Questions for thought
WHAT
to test?
Skills
Speaking Reading
Listening Writing
Knowledge
Language
Grammar
lexis
semantics
syntax
morphology
Reading
WHAT
to test? Reading aloud Silent reading
pronunciation
Stresses & intonation• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Listening
WHAT
to test?
• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Grammar
WHAT
to test?
Syntax (sentence structure)
Mechanics: e.g Punctuation, capital letter, comma, etc..
Rules of grammar
pronouns
nouns
verbs
subject & verb agreement
prepositions
tenses
adjectives adverbscomparatives
Speaking
WHAT
to test?
Language
Presentation skills
Content / ideas
Pronunciation
Organization
quantity quality
clarity
accuracy
appropriacy format
vocabulary Sentence complexity
syntax grammar
fluency confidence
Eye contact style
Writing
WHAT
to test?
LanguageContent / ideas
Organization
quantityquality
clarity
accuracy
appropriacy
Paraphrasing
voc Sentence complexity
syntax
Topic sentence Supportive sentences
compoundSimple complex
WHAT
to test?
Skills
Speaking Reading
Listening Writing
Knowledge
Language
Grammar
lexis
semantics
syntax
morphology
Reading
WHAT
to test? Reading aloud Silent reading
pronunciation
Stresses & intonation• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Listening
WHAT
to test?
• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Grammar
WHAT
to test?
Syntax (sentence structure)
Mechanics: e.g Punctuation, capital letter, comma, etc..
Rules of grammar
pronouns
nouns
verbs
subject & verb agreement
prepositions
tenses
adjectives adverbscomparatives
Speaking
WHAT
to test?
Language
Presentation skills
Content / ideas
Pronunciation
Organization
quantity quality
clarity
accuracy
appropriacy format
vocabulary Sentence complexity
syntax grammar
fluency confidence
Eye contact style
Writing
WHAT
to test?
LanguageContent / ideas
Organization
quantityquality
clarity
accuracy
appropriacy
Paraphrasing
voc Sentence complexity
syntax
Topic sentence Supportive sentences
compoundSimple complex
What makes test questions difficult?
THINKING LEVELS
Designing test questions in accordance with the
levels of thinking
Bloom’s taxonomy
Old version, 1956, Benjamin Bloom
New version, 1990’s , Lorin Anderson
BLOOM’S TAXONOMY.
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Arrange, define, list, memorise, name, relate, recall, repeat, state
Classify, describe, discuss, explain, express, recognise, identify, report
Apply, choose, demonstrate, illustrate, solve, use, interpret
Analyse, calculate, categorise, criticise, contrast, distinguish
Arrange, collect, compose, plan, construct, create, design, develop
Argue, assess, justify, judge, rate, support, value, evaluate
Useful verbs Sample questions stems
TellListDescribeRelateLocateWriteFindStatename
• What happened after….?• How many…..?• Who was it that……..?• Can you name the …….?• Describe what happened at…..?• Who spoke to…..?• Can you tell why….?• Find the meaning of …..?• What is …….?• Which is true or false…….?
Bloom’s taxonomy – Knowledge level
Useful verbs Sample questions stems
ExplainInterpretOutlineDiscussDistinguishPredictRestateCompareTranslateDescribe
• Can you write in your own words….?
• Can you write a brief outline….?• What do you think could happen
next….?• Who do you think….?• What was the main idea…..?• Who was the key character…..?• Can you distinguish between….?• Can you provide an example of …..?• What do you mean ….?• Can you provide a definition for….?
Bloom’s taxonomy – Comprehension level
Useful verbs Sample questions stems
Solve ShowUseillustrateConstructCompleteExamineClassify
• Do you know another instance where….?
• Could this have happened in ….?• Can you group by charateristics such
as….?• What factors would you change if…..?• What questions would you ask of…..?• From the information given, can you
develop a set of instructions about……?
• Would this information be useful if you had….?
Bloom’s taxonomy – Application level
Useful verbs Sample questions stems
AnalyseExamineCompareContrastInvestigateCategoriseIdentifyExplainSeparate
• Which events could have happened….?
• What do you see as other possible outcomes…?
• How was this similar to….?• Can you explain what must have
happened when…?• What are some of the problems of ….?• Can you distinguish between….?• What were some of the motives
behind….?
Bloom’s taxonomy – Analysis level
Useful verbs Sample questions stems
CreateInventComposePredictPlanConstructDesignImaginePropose devise
• Can you design a…..to….?• Can you see a possible solution to ….?• If you had access to all resources how
would you deal with….?• What would happen if …..?• How many ways can you……?• Can you write a new recipe for a tasty
dish?• Why not compose a song about…..?
Bloom’s taxonomy – Synthesis level
Useful verbs Sample questions stems
JudgeSelectChooseDecideJustifyDebateVerifyArgueRecommenddetermine
• Is there a better solution to ….• Judge the value of….• Can you defend your position
about…..?• Do you think……is a good or bad
thing?• What changes to ……would you
recommend?• Do you believe….?• Are you a …..person?• How would you feel if….?• How effective are…..?• What do you think about…….?
Bloom’s taxonomy – Evaluation level
BLOOM’S TAXONOMY.
Skills Subskills Question types Levels
BLOOM’S TAXONOMY.
Skills Subskills Question types Levels
Skills Sub-skills Question types Question levels
Reading Identifying main facts & details (FD)
Recognizing cause & effects relationship (CE)
Inferring meaning from contextual context ( ICC)
Predicting outcomes (PO) Sequence of events (SOE) Evaluating writer’s attitude &
view
True/ false Re-arrangement Structured Open-ended (subjective
response, free writing) MCQ (wh-question,
matching, completion with options)
Cloze procedure
Listening Identifying facts & details relate cause & effect identify sequence of events Inferring meaning from
contextual clues. Predicting out comes Inferring speaker’s attitude &
emotion (sad, happy, worried) Inferring relationship between
speakers.
MCQ with pictures or without pictures.
Fill in the blanks/ completion item.
Labelling. Matching True / false. sequencing
BARRETT’S TAXONOMY.
Table of Specifications
Question levels
Bloom’s(Know,Comp, App, Ana, Syn, Eva)
Barrett’s (LC, RO, IC, Eva, Appr)
Skills & subskills
Mark allocation
Question
Sections/numbers
Time allocation
Curriculum specs
knowledge
Question types
(MCQ, T/F, gap-fill, matching, sequencing, open-ended)
LSRW
LOGO
Dinh Cong Triet