writing mcqs susan chamberlain dept of obs/gyn queen’s university queen’s university

63
Writing MCQs Writing MCQs Susan Chamberlain Susan Chamberlain Dept of OBS/GYN Dept of OBS/GYN Queen’s University Queen’s University

Upload: julian-hubbard

Post on 23-Dec-2015

220 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Writing MCQsWriting MCQs

Susan ChamberlainSusan Chamberlain

Dept of OBS/GYNDept of OBS/GYN

Queen’s UniversityQueen’s University

Page 2: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Writing MCQsWriting MCQs

At the end of this session, the participant will At the end of this session, the participant will be able to be able to

Identify different types of MCQ itemsIdentify different types of MCQ items Describe the structure of an MCQDescribe the structure of an MCQ List the technical flaws common to MCQ’sList the technical flaws common to MCQ’s write one well-structured MCQ testing write one well-structured MCQ testing

appropriate contentappropriate content

Page 3: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Writing MCQsWriting MCQs

At the end of this session, the participant will At the end of this session, the participant will be able to be able to

Identify different types of MCQ itemsIdentify different types of MCQ items Describe the structure of an MCQDescribe the structure of an MCQ List the technical flaws common to MCQ’sList the technical flaws common to MCQ’s write one well-structured MCQ testing write one well-structured MCQ testing

appropriate contentappropriate content All of the aboveAll of the above

Page 4: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Check-listMCQ Check-list

Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it

Page 5: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Why do we test?Why do we test?

Communicate to the students what material Communicate to the students what material is important (formative vs summative)is important (formative vs summative)

Motivate students to studyMotivate students to study Identify areas of deficiency in need of Identify areas of deficiency in need of

remediation or further learningremediation or further learning Determine final grades or make promotion Determine final grades or make promotion

decisionsdecisions Identify areas where the course/curriculum Identify areas where the course/curriculum

is weakis weak

Page 6: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What Should Be Tested?What Should Be Tested?

Exam content should match course Exam content should match course objectivesobjectives

Important topics should be weighted more Important topics should be weighted more heavily than less important topicsheavily than less important topics

The testing time devoted to each topic The testing time devoted to each topic should reflect the relative importance of the should reflect the relative importance of the topicstopics

The sample of items should be The sample of items should be representative of the instructional goalsrepresentative of the instructional goals

Page 7: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What Should Be Tested?What Should Be Tested?

SamplingSampling““Performance on the sample provides a basis Performance on the sample provides a basis for estimating achievement in the broader for estimating achievement in the broader domain that is actually of interest.”domain that is actually of interest.”

Questions sample: Questions sample: – TOPICSTOPICS– SKILLS ie determining diagnosis, next step in SKILLS ie determining diagnosis, next step in

management, etc.management, etc.

Page 8: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ ‘Items’MCQ ‘Items’

Should test important information – either Should test important information – either common or life-threateningcommon or life-threatening

Should be well-structuredShould be well-structured– Flaws benefit test-wise examineesFlaws benefit test-wise examinees– Irrelevant difficulty does not test examinees’ Irrelevant difficulty does not test examinees’

specific knowledgespecific knowledge

Page 9: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Anatomy of an MCQ ItemAnatomy of an MCQ Item

StemStem Lead-inLead-in OptionsOptions

– One correct answerOne correct answer– DistractorsDistractors

Page 10: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Types of MCQ’sTypes of MCQ’s

True/False familyTrue/False family Assess recall of isolated factsAssess recall of isolated facts

– Stems must be clear and unambiguousStems must be clear and unambiguous– Options must be absolutely true or false or Options must be absolutely true or false or

examinee must decide examinee must decide how truehow true options are options are

Generally not recommended – usually have Generally not recommended – usually have subtle flawssubtle flaws

Page 11: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Types of MCQ’s: True/FalseTypes of MCQ’s: True/False

Which of the following statements is true regarding Which of the following statements is true regarding contraception?contraception?

A)A) oral contraceptives should not be initiated until six weeks oral contraceptives should not be initiated until six weeks after therapeutic abortion to avoid the risk of after therapeutic abortion to avoid the risk of hypercoagulabilityhypercoagulability

B)B) an intrauterine device can provide good contraceptive an intrauterine device can provide good contraceptive protection 5 years after insertionprotection 5 years after insertion

C)C) oral contraceptives should not be prescribed to individuals oral contraceptives should not be prescribed to individuals under the age of 15 without parental consentunder the age of 15 without parental consent

D)D) lesbian women do not require counseling about lesbian women do not require counseling about contraception or sexually transmitted diseasescontraception or sexually transmitted diseases

E)E) oral contraceptives should not be prescribed for women overoral contraceptives should not be prescribed for women over the age of 35the age of 35

Page 12: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Types of MCQ’sTypes of MCQ’s

One-Best-Answer family (A type)One-Best-Answer family (A type) Options may be diagrammed:Options may be diagrammed:

D C A E BD C A E B

Correct answer is the Correct answer is the most likelymost likely correct correct Better application of knowledge, integration, Better application of knowledge, integration,

synthesis and judgmentsynthesis and judgment

Least correct Most correct

Page 13: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Types of MCQ’s: Types of MCQ’s: One-Best-Answer familyOne-Best-Answer family

A 24 year old university student came to see you because she is A 24 year old university student came to see you because she is missing several days of school each month because of severe missing several days of school each month because of severe dysmenorrhea unresponsive to the OCP and NSAIDS. She also dysmenorrhea unresponsive to the OCP and NSAIDS. She also complains of deep dyspareunia and has a tender uterosacral complains of deep dyspareunia and has a tender uterosacral nodule on the pelvic examination. Which one of the following nodule on the pelvic examination. Which one of the following conditions is most likely responsible for her symptoms?conditions is most likely responsible for her symptoms?

A)A) Chronic pelvic inflammatory diseaseChronic pelvic inflammatory diseaseB)B) EndometriosisEndometriosisC)C) AdenomyosisAdenomyosisD)D) FibroidsFibroids

Page 14: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Check-listMCQ Check-list

Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it

Page 15: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Technical FlawsMCQ Technical Flaws

Issues Related to Test-Wiseness Issues Related to Test-Wiseness Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty

Page 16: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Technical FlawsMCQ Technical Flaws

Issues Related to Test-WisenessIssues Related to Test-Wiseness– provide clues to the right answer provide clues to the right answer – examinee can eliminate options based on flaws examinee can eliminate options based on flaws

and not on knowledgeand not on knowledge

Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty

Page 17: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wisenessItem Flaws: Test-wiseness

Grammatical cuesGrammatical cues Logical cuesLogical cues Absolute termsAbsolute terms Long correct answerLong correct answer Word repeatsWord repeats Convergence strategyConvergence strategy

Page 18: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wisenessItem Flaws: Test-wiseness

Grammatical cuesGrammatical cues One or more distractors don’t follow One or more distractors don’t follow

grammatically from the stem grammatically from the stem

Page 19: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wisenessItem Flaws: Test-wiseness

Grammatical cuesGrammatical cuesA 24 year old patient presents with a one year A 24 year old patient presents with a one year

history of dysmenorrhea, mid-cycle pain, history of dysmenorrhea, mid-cycle pain, dyspareunia, and painful bowel movements with dyspareunia, and painful bowel movements with her periods. The most likely diagnosis is:her periods. The most likely diagnosis is:

A)A) Fibroid uterusFibroid uterus

B)B) AdenomyosisAdenomyosis

C)C) EndometriosisEndometriosis

D)D) Psychosomatic conditionPsychosomatic condition

Page 20: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Logical cuesLogical cues A subset of the options are collectively A subset of the options are collectively

exhaustiveexhaustive

Page 21: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Logical cuesLogical cues Arterial blood gases in a normal pregnant Arterial blood gases in a normal pregnant

woman, compared to her non-pregnant state may woman, compared to her non-pregnant state may show which of the following?show which of the following?

A)A) Decreased pODecreased pO22

B)B) Increased pCOIncreased pCO22

C)C) Decreased pCODecreased pCO22

D)D) Decreased pHDecreased pH

E)E) No difference in pCONo difference in pCO22

Page 22: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Logical cuesLogical cues One or more distractors don’t follow One or more distractors don’t follow

logicallylogically from the stem from the stem

Page 23: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Logical cuesLogical cuesAn important tool for assessing fetal well-being is An important tool for assessing fetal well-being is

the Non Stress test. A well baby demonstrates the Non Stress test. A well baby demonstrates the following fetal heart rate pattern:the following fetal heart rate pattern:

A)A) A baseline heart rate above 120 bpmA baseline heart rate above 120 bpm

B)B) Evidence of fetal movementEvidence of fetal movement

C)C) Baseline variabilityBaseline variability

D)D) Two fetal heart rate accelerations in 20 minutesTwo fetal heart rate accelerations in 20 minutes

E)E) Fetal breathing movementsFetal breathing movements

Page 24: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Absolute termsAbsolute terms Always, neverAlways, never Not a problem if stem is long and lead-in Not a problem if stem is long and lead-in

asks a precise questionasks a precise question

Page 25: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Absolute termsAbsolute terms

In patients with a previous Cesarean section:In patients with a previous Cesarean section:A)A) Vaginal birth is NEVER an optionVaginal birth is NEVER an option

B)B) Repeat Cesarean section is less dangerous to the Repeat Cesarean section is less dangerous to the mothermother

C)C) The uterine scar may rupture during labourThe uterine scar may rupture during labour

D)D) Care should ALWAYS be supervised by an Care should ALWAYS be supervised by an obstetricianobstetrician

E)E) No further pregnancies should EVER be consideredNo further pregnancies should EVER be considered

Page 26: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Long correct answerLong correct answer Correct answer is longer, more specific, or Correct answer is longer, more specific, or

more complete than other optionsmore complete than other options

Page 27: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Long correct answerLong correct answer Which of the following problems in vaginal Which of the following problems in vaginal

breech delivery is most likely to cause serious breech delivery is most likely to cause serious long-term complications?long-term complications?

A)A) Failure to descendFailure to descendB)B) Cord prolapseCord prolapseC)C) Nuchal armsNuchal armsD)D) Entrapment of the after coming head with an Entrapment of the after coming head with an

incompletely dilated cervixincompletely dilated cervixE)E) 1 minute Apgar score of 41 minute Apgar score of 4

Page 28: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wisenessItem Flaws: Test-wiseness

Word repeatsWord repeats A word or phase is included in the stem and A word or phase is included in the stem and

in the correct answerin the correct answer

Page 29: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wisenessItem Flaws: Test-wiseness

Word repeatsWord repeats

A 24 year old with a previous history of chronic A 24 year old with a previous history of chronic PID presents with primary infertility. Select the PID presents with primary infertility. Select the procedure that would be most appropriate to procedure that would be most appropriate to evaluate the status of her evaluate the status of her tubestubes and pelvic and pelvic anatomyanatomy

A)A) Post-coital testPost-coital test

B)B) Endometrial biopsyEndometrial biopsy

C)C) Laparoscopy with dye insufflation of the Laparoscopy with dye insufflation of the fallopian fallopian tubestubes

D)D) HysterosalpingogramHysterosalpingogram

Page 30: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Convergence strategyConvergence strategy The correct answer includes the most The correct answer includes the most

elements in common with the other optionselements in common with the other options

Page 31: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Item Flaws: Test-wiseness Item Flaws: Test-wiseness

Convergence strategyConvergence strategyNormal bladder neck mobility is measured by:Normal bladder neck mobility is measured by:

A)A) A Q-tip angle measurement under 40 degreesA Q-tip angle measurement under 40 degrees

B)B) A Q-tip angle measurement under 30 degreesA Q-tip angle measurement under 30 degrees

C)C) A bladder neck ultrasound angle less than 30 A bladder neck ultrasound angle less than 30 degreesdegrees

D)D) A Q-tip angle measurement under 20 degreesA Q-tip angle measurement under 20 degrees

Page 32: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Technical FlawsMCQ Technical Flaws

Issues Related to Test-Wiseness Issues Related to Test-Wiseness

Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty– do not assess knowledge of intended topicdo not assess knowledge of intended topic

Page 33: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Options are long, complicated or doubleOptions are long, complicated or double Extraneous details in stemExtraneous details in stem Numeric data not stated consistentlyNumeric data not stated consistently Vague frequency termsVague frequency terms Non-parallel language in optionsNon-parallel language in options Stems are tricky or unnecessarily complicatedStems are tricky or unnecessarily complicated ‘‘None of the above’None of the above’

Page 34: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Options are long, complicated or doubleOptions are long, complicated or double

Extraneous details in stemExtraneous details in stem Shifts difficulty from content knowledge to Shifts difficulty from content knowledge to

speed readingspeed reading

Page 35: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Options are long, complicated or doubleOptions are long, complicated or double Mrs. Brown is a 70 year old woman who had a stroke 6 months Mrs. Brown is a 70 year old woman who had a stroke 6 months

ago and is still impaired. Her daughter tells you that her ago and is still impaired. Her daughter tells you that her mother’s recent urinary incontinence makes her care mother’s recent urinary incontinence makes her care unmanageable. She is seeking permanent placement. Mrs. unmanageable. She is seeking permanent placement. Mrs. Brown complains of urinary frequency, nocturia, and urge Brown complains of urinary frequency, nocturia, and urge incontinence. Which of the following statements is true?incontinence. Which of the following statements is true?

A)A) Neurological conditions are involved in the majority of women with Neurological conditions are involved in the majority of women with symptoms like those presented by Mrs. Brownsymptoms like those presented by Mrs. Brown

B)B) Diabetes has a greater impact on quality of life than Mrs. Brown’s Diabetes has a greater impact on quality of life than Mrs. Brown’s condition.condition.

C)C) The pharmacological blockage of muscarinic receptors in the detrusor The pharmacological blockage of muscarinic receptors in the detrusor muscle controls symptoms compatible with Mrs. Brown’s conditionmuscle controls symptoms compatible with Mrs. Brown’s condition

D)D) Overactive bladder is defined as urinary incontinence accompanied by Overactive bladder is defined as urinary incontinence accompanied by or immediately preceded by urgencyor immediately preceded by urgency

E)E) Alpha stimulants are effective in treating this conditionAlpha stimulants are effective in treating this condition

Page 36: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Extraneous details in stemExtraneous details in stem You see in your office a 24 year old woman for her routine postpartum You see in your office a 24 year old woman for her routine postpartum

visit, at 6 weeks. Her baby is irritable and cries “all the time’ visit, at 6 weeks. Her baby is irritable and cries “all the time’ (including during her visit.) As she is changing the diaper during her (including during her visit.) As she is changing the diaper during her visit, you notice that the child has a very red bum but upon visit, you notice that the child has a very red bum but upon questioning, she tells you she did not notice anything abnormal. Her questioning, she tells you she did not notice anything abnormal. Her partner is away in Afghanistan and she does not have a lot of friends, partner is away in Afghanistan and she does not have a lot of friends, as they just recently were posted in Kingston. She states that she has a as they just recently were posted in Kingston. She states that she has a hard time getting dressed in the morning, cries easily and feels like she hard time getting dressed in the morning, cries easily and feels like she does not love her child. She denies any medical or psychiatric history. does not love her child. She denies any medical or psychiatric history. Her exam is essentially normal. She weighs 2 kg less than at her first Her exam is essentially normal. She weighs 2 kg less than at her first antenatal visit at 10 weeks. You most likely clinical diagnosis is:antenatal visit at 10 weeks. You most likely clinical diagnosis is:

A)A) Postpartum hypothyroidismPostpartum hypothyroidism

B)B) HyperthyroidismHyperthyroidism

C)C) Postpartum bluesPostpartum blues

D)D) Postpartum depressionPostpartum depression

E)E) Psychotic depressionPsychotic depression

Page 37: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant Difficulty Irrelevant Difficulty

Numeric DataNumeric Data Not stated consistentlyNot stated consistently Mixed formatMixed format Illogical orderIllogical order

Page 38: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant Difficulty Irrelevant Difficulty

Numeric Data not stated consistentlyNumeric Data not stated consistently Following a second episode of salpingitis, what Following a second episode of salpingitis, what

is the likelihood that a woman is infertile?is the likelihood that a woman is infertile?A)A) Less than 20%Less than 20%

B)B) 20 – 30 %20 – 30 %

C)C) Greater than 50 %Greater than 50 %

D)D) 90%90%

E)E) 75%75%

Page 39: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant Difficulty Irrelevant Difficulty

Vague frequency termsVague frequency terms Often, usually, rarely, frequently Often, usually, rarely, frequently Not consistently definedNot consistently defined

*may be used in stem if otherwise carefully *may be used in stem if otherwise carefully constructed unambiguous questionconstructed unambiguous question

Page 40: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Vague frequency termsVague frequency terms

Gonorrhea isGonorrhea is

A) sometimes asymptomaticA) sometimes asymptomatic

B) treated with cefixime 450 mg orallyB) treated with cefixime 450 mg orally

C) caused by a gram positive intracellular C) caused by a gram positive intracellular diplococcusdiplococcus

D) diagnosed by a DNA testD) diagnosed by a DNA test

E) usually associated with vulvar ulcersE) usually associated with vulvar ulcers

Page 41: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Non-parallel language in optionsNon-parallel language in options Non-logical orderNon-logical order

Page 42: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Non-parallel language in optionsNon-parallel language in optionsIn a vaccine trial, 200 2-year-old boys were given a vaccine In a vaccine trial, 200 2-year-old boys were given a vaccine

against a certain disease and then monitored for five against a certain disease and then monitored for five years for occurrence of the disease. Of this group, 85% years for occurrence of the disease. Of this group, 85% never contracted the disease. Which of the following never contracted the disease. Which of the following statements concerning these results is correct?statements concerning these results is correct?

A)A) No conclusion can be drawn, since no follow-up was No conclusion can be drawn, since no follow-up was made of non-vaccinated childrenmade of non-vaccinated children

B)B) The number of cases (ie 30 cases over five year) is The number of cases (ie 30 cases over five year) is too small for statistically meaningful conclusionstoo small for statistically meaningful conclusions

C)C) No conclusions can be drawn because the trial No conclusions can be drawn because the trial involved only boysinvolved only boys

D)D) Vaccine efficacy (%) is calculated as 85-15/100Vaccine efficacy (%) is calculated as 85-15/100

Page 43: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

““None of the above”None of the above” Problematic where judgment is involved or Problematic where judgment is involved or

if options not absolutely true or falseif options not absolutely true or false Turns question into true/false questionTurns question into true/false question Confuses very knowledgeable studentsConfuses very knowledgeable students

Page 44: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

““None of the above”None of the above”

The main reason that fertility is reduced in a The main reason that fertility is reduced in a woman in her early 40’s is:woman in her early 40’s is:

A)A) Irregular cycles make it hard to time intercourseIrregular cycles make it hard to time intercourse

B)B) Chromosomal damage adversely affects oocytes’ Chromosomal damage adversely affects oocytes’ genetic complementgenetic complement

C)C) Male erectile difficulties increaseMale erectile difficulties increase

D)D) Hormone imbalance leads to poor endometrial Hormone imbalance leads to poor endometrial developmentdevelopment

E)E) None of the aboveNone of the above

Page 45: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Irrelevant DifficultyIrrelevant Difficulty

Stems are tricky or unnecessarily complicatedStems are tricky or unnecessarily complicated Arrange the parents of the following children with Down’s syndrome Arrange the parents of the following children with Down’s syndrome

in order of the highest to lowest risk of recurrence. Assume that the in order of the highest to lowest risk of recurrence. Assume that the maternal age in all cases is 22 years and that a subsequent pregnancy maternal age in all cases is 22 years and that a subsequent pregnancy occurs within 5 years. The karyotypes of the daughters are:occurs within 5 years. The karyotypes of the daughters are:

I: 46, XX, –14,+T (14q21q) patI: 46, XX, –14,+T (14q21q) patII: 46, XX, –14,+T (14q21q) de novoII: 46, XX, –14,+T (14q21q) de novoIII: 46, XX, –14,+T (14q21q) matIII: 46, XX, –14,+T (14q21q) matIV: 46, XX, –21,+T (14q21q) patIV: 46, XX, –21,+T (14q21q) patV: 46, XX, –21,+T (21q21q) (parents not typed)V: 46, XX, –21,+T (21q21q) (parents not typed)

A)A) III, IV, I, V, IIIII, IV, I, V, IIB)B) IV, III, V, 1, IIIV, III, V, 1, IIC)C) III, I, IV, V, IIIII, I, IV, V, IID)D) I, V, III, I, V, III, V, III, I, V, IIE)E) III, IV, I, II, VIII, IV, I, II, V

Page 46: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Check-listMCQ Check-list

Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it

Page 47: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What to test?What to test?

Items classified by cognitive processes required to Items classified by cognitive processes required to answer the question:answer the question:

Recall (Memory)– tests knowledge of isolated factsRecall (Memory)– tests knowledge of isolated facts Interpretation (Comprehension)– review information Interpretation (Comprehension)– review information

to reach conclusion, eg diagnosisto reach conclusion, eg diagnosis Problem solving (Reasoning) – situation requires Problem solving (Reasoning) – situation requires

action, eg next step in managementaction, eg next step in management

– Process depends on background of traineeProcess depends on background of trainee

Page 48: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What to test?What to test?

Simpler classification based on task:Simpler classification based on task: Application of knowledge item Application of knowledge item ****

– Reach conclusionsReach conclusions– Make predictionsMake predictions– Select course of actionSelect course of action

Recall itemRecall item– Test knowledge of isolated factsTest knowledge of isolated facts

Page 49: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What to test?What to test?

Test application of knowledge using clinical Test application of knowledge using clinical vignettesvignettes– Adds ‘face-validity’Adds ‘face-validity’– Usually selects more important, less trivialUsually selects more important, less trivial– Identifies student who’ve memorized but do not Identifies student who’ve memorized but do not

understandunderstand

BUT experience in patient care should not BUT experience in patient care should not be necessarybe necessary

Page 50: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

What to test - summaryWhat to test - summary

Focus item on key concepts and principles that are Focus item on key concepts and principles that are essential information (without access to essential information (without access to references) for all examinees to understandreferences) for all examinees to understand

Test material that is relevant to learning in clinical Test material that is relevant to learning in clinical clerkships and beyondclerkships and beyond

Avoid items that Avoid items that onlyonly require recall of isolated require recall of isolated factsfacts

Avoid esoteric or interesting topics that are not Avoid esoteric or interesting topics that are not essentialessential

Page 51: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Check-listMCQ Check-list

Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it

Page 52: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Writing the QuestionsWriting the Questions

Construct stem –include all necessary Construct stem –include all necessary information to arrive at the right answerinformation to arrive at the right answer

Choose lead-in – pose a clear questionChoose lead-in – pose a clear question Choose distractorsChoose distractors

Page 53: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Stem*Stem*

Usually a clinical scenario Usually a clinical scenario Clear, unambiguousClear, unambiguous Should be long relative to optionsShould be long relative to options Includes all pertinent informationIncludes all pertinent information

– Patient’s age/genderPatient’s age/gender– Clinical settingClinical setting– ComplaintComplaint– Other important info – history, physical Other important info – history, physical

findings, test resultsfindings, test results

Page 54: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Lead-in*Lead-in*

Follows stemFollows stem In the form of a questionIn the form of a question Should relate to the stemShould relate to the stem Should be answerable without looking at Should be answerable without looking at

optionsoptions– ““Cover the Options test”Cover the Options test”

Page 55: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

DistractorsDistractors

Each should be selected by some, therefore Each should be selected by some, therefore all plausible, none obviously incorrectall plausible, none obviously incorrect

3 or 3 or 44 choices choices– Common misconceptionsCommon misconceptions– Faulty reasoningFaulty reasoning

Page 56: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

DistractorsDistractors

Homogeneous in content to the correct answerHomogeneous in content to the correct answer Should be in same category as correct answer– Should be in same category as correct answer–

diagnosis, treatment, diagnostic testdiagnosis, treatment, diagnostic test Clearly incorrect or inferior to the correct answerClearly incorrect or inferior to the correct answer Plausible and attractive to the uninformedPlausible and attractive to the uninformed Similar to the correct answer in construction and Similar to the correct answer in construction and

length length Grammatically consistent and logically compatible Grammatically consistent and logically compatible

with the stemwith the stem

Page 57: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

General Guidelines for Item General Guidelines for Item ConstructionConstruction Include as much of the item as possible in the Include as much of the item as possible in the

stems: Long stems, short optionsstems: Long stems, short options Avoid negatively phrased items (Avoid negatively phrased items (exceptexcept or or notnot in in

the lead-in). If you must, use only with short the lead-in). If you must, use only with short optionsoptions

Avoid writing question of the form:Avoid writing question of the form:– Which of the following statements is correct?Which of the following statements is correct?

– Each of the following statements is correct EXCEPTEach of the following statements is correct EXCEPT» Unfocussed Unfocussed

» Heterogeneous optionsHeterogeneous options

Page 58: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

““Top-down” versus “Bottom-up”Top-down” versus “Bottom-up”

Top-DownTop-Down Knowledge or recall question Knowledge or recall question Cite a disease and then ask what patient Cite a disease and then ask what patient

findings are expectedfindings are expected Structured like textbooksStructured like textbooks

Clinically backwardClinically backward

Page 59: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

““Top-down” versus “Bottom-up”Top-down” versus “Bottom-up”

Bottom-upBottom-up Application of knowledgeApplication of knowledge Gives findings and asks examinee to Gives findings and asks examinee to

indicate the diseaseindicate the disease Examinees need to be able to synthesize Examinees need to be able to synthesize

information from several pages of a information from several pages of a textbook (or notes) to answer these textbook (or notes) to answer these questionsquestions

Clinically realisticClinically realistic

Page 60: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Guidelines for Clinical SciencesGuidelines for Clinical Sciencesie Clerkship and beyond…ie Clerkship and beyond… Test application of knowledge using clinical Test application of knowledge using clinical

vignettes to pose medical decisions in vignettes to pose medical decisions in patient care situationspatient care situations

Focus items on common or potentially Focus items on common or potentially catastrophic problemscatastrophic problems

Pose clinical decisions that would be Pose clinical decisions that would be expected of a successful examineeexpected of a successful examinee

Avoid clinical situations that would be Avoid clinical situations that would be handled by a (sub)specialisthandled by a (sub)specialist

Page 61: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

MCQ Check-listMCQ Check-list

Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it

Page 62: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

Basic Rules for MCQ’s (A type)*Basic Rules for MCQ’s (A type)*

Focus item on important conceptFocus item on important concept Item should assess application of Item should assess application of

knowledge, not recall of an isolated factknowledge, not recall of an isolated fact Stem must pose clear question – ‘Cover the Stem must pose clear question – ‘Cover the

options’ testoptions’ test All distractors should be homogeneousAll distractors should be homogeneous Avoid technical item flawsAvoid technical item flaws

Page 63: Writing MCQs Susan Chamberlain Dept of OBS/GYN Queen’s University Queen’s University

ReferenceReference

Constructing Written Test Questions For the Constructing Written Test Questions For the Basic and Clinical SciencesBasic and Clinical Sciences– Case & Swanson, NBME 1996Case & Swanson, NBME 1996