dr. shama mashhood

28
1

Upload: december

Post on 01-Feb-2016

56 views

Category:

Documents


0 download

DESCRIPTION

Dr. Shama Mashhood. Medical Educationist & Coordinator Question Review Committee. “Multiple Choice Questions- One best type”. Overview of the Presentation. Strengths and Limitations of one best type MCQs. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Dr. Shama Mashhood

11

Page 2: Dr. Shama Mashhood

22

Dr. Shama MashhoodDr. Shama Mashhood

Medical Educationist Medical Educationist && Coordinator Coordinator Question Review CommitteeQuestion Review Committee

Page 3: Dr. Shama Mashhood

33

““Multiple Choice Multiple Choice Questions- One best Questions- One best type”type”

Page 4: Dr. Shama Mashhood

44

Overview of the Overview of the PresentationPresentation

Strengths and Limitations of one best type Strengths and Limitations of one best type MCQs.MCQs.

Characteristics of good MCQs Items. Characteristics of good MCQs Items.

Checklist for review of MCQs.Checklist for review of MCQs.

Page 5: Dr. Shama Mashhood

55

STRENGTH OF MCQsSTRENGTH OF MCQs

ValidValid ReliableReliable StandardizedStandardized ObjectiveObjective Sample broad contentSample broad content Easy to score & analyze.Easy to score & analyze. Item analysis, difficulty & discrimination Item analysis, difficulty & discrimination Can test various levels of cognitionCan test various levels of cognition

Page 6: Dr. Shama Mashhood

66

LIMTATIONS OF MCQsLIMTATIONS OF MCQs

Difficult to prepare . Difficult to prepare .

Time consuming.Time consuming.

Assess recall (If carefully not constructed) Assess recall (If carefully not constructed)

Page 7: Dr. Shama Mashhood

77

Higher order problem solving and decision Higher order problem solving and decision making abilitymaking ability

Scoring can be affected by reading ability.Scoring can be affected by reading ability.

Difficult to determine why candidate Difficult to determine why candidate select wrong response select wrong response

Page 8: Dr. Shama Mashhood

88

ComponentsComponents A 68 years old man presents with colicky A 68 years old man presents with colicky abdominal pain. Within three days of admission to abdominal pain. Within three days of admission to the hospital, he becomes jaundiced and develops the hospital, he becomes jaundiced and develops high-grade fluctuating pyrexia.high-grade fluctuating pyrexia.

The most likely diagnosis in this patient isThe most likely diagnosis in this patient is

A. infective hepatitisA. infective hepatitisB. cirrhosis of liverB. cirrhosis of liverC. a stone impacted in the common bile ductC. a stone impacted in the common bile ductD. carcinoma of the head of pancreasD. carcinoma of the head of pancreasE. an intra hepatic abscessE. an intra hepatic abscess

KEY: CKEY: C

Question Stem

Lead in statemen

t

Option

Page 9: Dr. Shama Mashhood

99

Item Shape?Item Shape?

Long stem with all essential information

Short Option

Short Option

Short Option

Short Option

Short Option

Page 10: Dr. Shama Mashhood

1010

Item Shape?Item Shape?Short item stem

Long Option

Long Option

Long Option

Long Option

Page 11: Dr. Shama Mashhood

1111

Item Shape?Item Shape?

Item Stem

Uneven option

Uneven option

Uneven option

Uneven option

Page 12: Dr. Shama Mashhood

1212

Characteristics of good Characteristics of good Item Item Identify Objectives, level and Identify Objectives, level and

domain of learning domain of learning Formation of table of Formation of table of

specification.specification. Simple recall to problem solving Simple recall to problem solving

questions.questions.

Page 13: Dr. Shama Mashhood

1313

GOOD ITEM STEMGOOD ITEM STEM

1.Must be clear, concise, straightforward 1.Must be clear, concise, straightforward and a complete question. and a complete question.

E.g. E.g. Multiple choice question isMultiple choice question is ValidValid ReliableReliable ObjectiveObjective FeasibleFeasible

Page 14: Dr. Shama Mashhood

1414

Recall Item:Recall Item:

e.g. e.g. What area is supplied with blood by the What area is supplied with blood by the posterior inferior cerebellar artery?posterior inferior cerebellar artery?

e.g. e.g. Which of the following presents asWhich of the following presents asChronic (longer than 3 months) airspace Chronic (longer than 3 months) airspace

disease on a chest radiograph?disease on a chest radiograph? Streptococcal pneumoniaStreptococcal pneumonia Adult respiratory distress syndromeAdult respiratory distress syndrome Pulmonary edemaPulmonary edema Pulmonary alveolar proteinosisPulmonary alveolar proteinosis

Page 15: Dr. Shama Mashhood

1515

Item with a vignette/case/scenario Item with a vignette/case/scenario measuring application of knowledge measuring application of knowledge e.g. e.g. A 35 year woman presented with a 4 A 35 year woman presented with a 4

month history of dysnea, low grade fever, month history of dysnea, low grade fever, cough and fatigue. Given the following chest cough and fatigue. Given the following chest radiograph, what is the most likely diagnosis? radiograph, what is the most likely diagnosis? Streptococcal pneumoniaStreptococcal pneumonia

Adult respiratory distress syndromeAdult respiratory distress syndrome Pulmonary edemaPulmonary edema Pulmonary alveolar proteinosisPulmonary alveolar proteinosis Streptococcal pneumoniaStreptococcal pneumonia

Page 16: Dr. Shama Mashhood

1616

3. Worded positively.3. Worded positively.

e.g.e.g. Which of the following best Which of the following best distinguishes small airway distinguishes small airway disease from interstitial lung disease from interstitial lung disease on chest CT?disease on chest CT?

Page 17: Dr. Shama Mashhood

1717

4. No negative phrasing b/c negative stem 4. No negative phrasing b/c negative stem less effective and difficult to understand. less effective and difficult to understand. e.g. “Which is not true" or “all of the e.g. “Which is not true" or “all of the following except”following except”

e.g.e.g. Which of the following is Which of the following is NOTNOT a a characteristic CT finding of small characteristic CT finding of small airway disease?airway disease?

Page 18: Dr. Shama Mashhood

1818

6. Must have a central idea which fits 6. Must have a central idea which fits grammaticallygrammatically with the option.with the option.

ee.g..g. If the stem is in past tense ,all If the stem is in past tense ,all options should be in past tense.options should be in past tense.

If answer demands for plural, then If answer demands for plural, then all options should be in plural. all options should be in plural.

Page 19: Dr. Shama Mashhood

1919

AvoidAvoid 7. Absolute terms e.g. ‘Always’, ‘Never’, ‘all’, ‘none’7. Absolute terms e.g. ‘Always’, ‘Never’, ‘all’, ‘none’ b/c few ideas or situations are absolute or universally true.b/c few ideas or situations are absolute or universally true.8. Terms cues for test wise examinees e.g. ‘may’, ‘could’, 8. Terms cues for test wise examinees e.g. ‘may’, ‘could’,

‘can’. (b/c anything is possible)‘can’. (b/c anything is possible)9. ‘Seldom’, ‘rarely’, ‘occasionally’, ‘sometimes’, ‘few’,& 9. ‘Seldom’, ‘rarely’, ‘occasionally’, ‘sometimes’, ‘few’,&

‘many'. (not uniformly understood)‘many'. (not uniformly understood)

10. Excessive verbiage10. Excessive verbiage

11. Pronoun ‘it, he, she & you’.11. Pronoun ‘it, he, she & you’.

Page 20: Dr. Shama Mashhood

2020

GOOD LEAD INGOOD LEAD IN

Focus on one aspect of the disease of the Focus on one aspect of the disease of the condition.condition.

e.g. Diagnosis, Managemente.g. Diagnosis, Management

Page 21: Dr. Shama Mashhood

2121

GOOD OPTIONSGOOD OPTIONS

Grammatically consistent with stemGrammatically consistent with stem

Similar to correct answer in construction and Similar to correct answer in construction and lengthlength

Homogenous in content e.g. all diagnosis, test, Homogenous in content e.g. all diagnosis, test, treatments.treatments.

Avoid ‘none of the above’ or ‘all of the Avoid ‘none of the above’ or ‘all of the above’.above’.

Page 22: Dr. Shama Mashhood

2222

Keep options in logical order e.g. Keep options in logical order e.g. numerical,chronological,alphabetical, numerical,chronological,alphabetical, ascending order.ascending order.

Change position of key responses.Change position of key responses.

Keep options independent and not Keep options independent and not overlapping.overlapping.

Page 23: Dr. Shama Mashhood

2323

Check listCheck list

Reviewing multiple choice itemsReviewing multiple choice items

Page 24: Dr. Shama Mashhood

2424

1.1. Is this the most appropriate type of item to Is this the most appropriate type of item to use?use?

2.2. Does each item stem present a meaningful Does each item stem present a meaningful problem?problem?

3.3. Are the item stems free of irrelevant Are the item stems free of irrelevant material?material?

4.4. Are the item stems stated in positive term (if Are the item stems stated in positive term (if possible)?possible)?

Page 25: Dr. Shama Mashhood

2525

55.If used has the negative wording been given .If used has the negative wording been given special emphasis (e.g. capitalized)special emphasis (e.g. capitalized)

66.Are the alternatives grammatically consistent .Are the alternatives grammatically consistent with the item stem?with the item stem?

77.Are the alternative answers brief and free of .Are the alternative answers brief and free of unnecessary words?unnecessary words?

88.Are the alternatives similar in length and form?.Are the alternatives similar in length and form?

Page 26: Dr. Shama Mashhood

2626

99.Is there only one correct or clearly best .Is there only one correct or clearly best answer?answer?

1010.Are the distracters plausible to low achievers?.Are the distracters plausible to low achievers?

11.11.Are the items free of verbal clues to the Are the items free of verbal clues to the answer?answer?

12.12.Are the verbal alternatives in alphabetical Are the verbal alternatives in alphabetical order?order?

Page 27: Dr. Shama Mashhood

2727

1313.Are numerical alternatives in numerical .Are numerical alternatives in numerical order?order?

14.14.Have Have none of the abovenone of the above and and all of the aboveall of the above been avoided (or used sparingly and been avoided (or used sparingly and appropriately)?appropriately)?

15.15.If revised, are the items still relevant to the If revised, are the items still relevant to the intended learning outcomes?intended learning outcomes?

16.16.Have the items been set aside for a time Have the items been set aside for a time before reviewing them?before reviewing them?

Page 28: Dr. Shama Mashhood

2828

QUESTIONS?