Some Syllables Some Syllables About SyllabiAbout Syllabi

Jess Mandel, M.D.Jess Mandel, M.D.Associate Professor of MedicineAssociate Professor of Medicine

Associate Dean for Undergraduate Associate Dean for Undergraduate Medical EducationMedical Education

UCSD School of MedicineUCSD School of MedicineSeptember, 2006September, 2006

Neal Cassady (1926-1968)

Neal Cassady (1926-1968)

“I want you to tell me absolutely everything about your life…explore every digression and blind alley and don’t leave a single event out…”

OutlineOutline

I. Essential components of a syllabusI. Essential components of a syllabus II. Decisions in syllabus designII. Decisions in syllabus design III. Components/stepsIII. Components/steps

Goals/objectivesGoals/objectives Selecting appropriate teaching methodsSelecting appropriate teaching methods Assessment/grading policiesAssessment/grading policies TextbooksTextbooks HandoutsHandouts

IV. ConclusionsIV. Conclusions

SyllabusSyllabus

Definition:Definition: An outline or a summary An outline or a summary of the main points of a text, lecture, of the main points of a text, lecture, or course of study. or course of study.

Etymology:Etymology: Medieval Latin, probably Medieval Latin, probably alteration (influenced by Greek alteration (influenced by Greek sullambaneinsullambanein, to put together) of , to put together) of Latin Latin sillybussillybus, parchment label, from , parchment label, from Greek Greek sillubossillubos. .

Why Is a Good Syllabus Why Is a Good Syllabus Important?Important?

First interaction between students and First interaction between students and instructorinstructor

Delineates meaning, relevance, and goals Delineates meaning, relevance, and goals of the courseof the course

Establishes instructor’s and students’ Establishes instructor’s and students’ responsibilities and expectations, i.e., is a responsibilities and expectations, i.e., is a contractcontract

Defines standards of evaluationDefines standards of evaluation Acquaints students with course logisticsAcquaints students with course logistics Contains critical course materials Contains critical course materials Developing a good syllabus requires Developing a good syllabus requires

reflection and analysis – helps course reflection and analysis – helps course organization and executionorganization and execution

Essential ComponentsEssential Components

Course informationCourse information Course description/objectivesCourse description/objectives Instructor informationInstructor information Texts, readings, materialsTexts, readings, materials Course calendar/scheduleCourse calendar/schedule Course policies (Course policies (gradinggrading, attendance, , attendance,

participation, missed participation, missed exams/assignments, disabilites, etc)exams/assignments, disabilites, etc)

StepsSteps1.1. ReflectionReflection2.2. Establish goals and objectivesEstablish goals and objectives3.3. Select appropriate teaching and learning Select appropriate teaching and learning

methodsmethods4.4. Pick assessment methods of studentsPick assessment methods of students5.5. Think about contact with studentsThink about contact with students6.6. Select textbooksSelect textbooks7.7. Assemble handoutsAssemble handouts8.8. Consider other resourcesConsider other resources9.9. Plan evaluation of the coursePlan evaluation of the course

1) Reflection1) Reflection Develop a well-Develop a well-

grounded rationale for grounded rationale for the coursethe course

Decide what students Decide what students should be able to do as should be able to do as a result of the course.a result of the course.

Think about Think about appropriate appropriate assessmentsassessments

Consider how learning Consider how learning can be activecan be active

Identify resourcesIdentify resources Start composing the Start composing the

syllabussyllabus

Edouard Gelhay, Reflection, 1881

The Course RationaleThe Course Rationale

The rationale for the The rationale for the course is key.course is key.

Must be well Must be well thought out and well thought out and well articulatedarticulated Impacts construction Impacts construction

of courseof course Critical for students Critical for students

to appreciate to appreciate relevancerelevance

What is obvious to What is obvious to profs not obvious to profs not obvious to studentsstudents

Stuart Davis, Blips and Ifs, 1964

Example – Course RationaleExample – Course Rationale

Optical Microscopy and Imaging for Biologists 2006Optical Microscopy and Imaging for Biologists 2006

Understanding the function or dysfunction of cells is Understanding the function or dysfunction of cells is crucial in the field of modern medicine. Scientists crucial in the field of modern medicine. Scientists working in the biomedical sciences have access to a working in the biomedical sciences have access to a wide range of tools to examine the cells in their wide range of tools to examine the cells in their different states. This offers new opportunities as well different states. This offers new opportunities as well as new problems. Understanding the principles of as new problems. Understanding the principles of different optical imaging methods is critical to different optical imaging methods is critical to obtaining reliable results. The course will cover basic obtaining reliable results. The course will cover basic concepts of light microscopy and introduce advanced concepts of light microscopy and introduce advanced techniques relevant to modern cell and molecular techniques relevant to modern cell and molecular biology. Students will gain extensive hands-on biology. Students will gain extensive hands-on experience with state-of-the-art equipment for optical experience with state-of-the-art equipment for optical imaging and computerized image analysis guided by imaging and computerized image analysis guided by experienced academic instructors.experienced academic instructors.

Example – Course RationaleExample – Course Rationale

One of the difficult things about teaching comedy One of the difficult things about teaching comedy is analyzing how exactly a phrase, a sentence, a is analyzing how exactly a phrase, a sentence, a joke is constructed as funny. Added to this is the joke is constructed as funny. Added to this is the need to fend off critics who say that explaining need to fend off critics who say that explaining why something is funny takes all the fun out of it. why something is funny takes all the fun out of it. We believe that this doesn't have to be the case, We believe that this doesn't have to be the case, that in fact the comedy in a text can be enhanced that in fact the comedy in a text can be enhanced by an astute, tight analysis.by an astute, tight analysis.

What to AvoidWhat to Avoid Vague intentionsVague intentions Teaching that has a Teaching that has a

tenuous relationship tenuous relationship to these intentionsto these intentions

Assessment that Assessment that bears little or no bears little or no relationship to eitherrelationship to either The above place The above place

students in a students in a frustrating high stakes frustrating high stakes guessing gameguessing game

Johann Fussli, The Nightmare, 1781

2) Getting More Specific – From 2) Getting More Specific – From Rationale to Goals and Rationale to Goals and

ObjectivesObjectives Goals –Goals – What the What the

course or institution course or institution seeks to achieveseeks to achieve

Learning objectivesLearning objectives – clear statements – clear statements of what students of what students should be able to do should be able to do as a result of a as a result of a course of study.course of study.

Stuart Davis, Owh! in San Pao , 1951

Where Do Objectives Come Where Do Objectives Come From?From?

Knowledge, skills and attitudes of the Knowledge, skills and attitudes of the writerwriter

ColleaguesColleagues PractitionersPractitioners Professional organizationsProfessional organizations Students – survey and assessmentStudents – survey and assessment

Example of ObjectivesExample of Objectives DescribeDescribe the the

complications of diabetescomplications of diabetes UnderstandUnderstand

environmental factors environmental factors that predispose children that predispose children to developing asthmato developing asthma

ObtainObtain a sexual history a sexual history from a patientfrom a patient

PerformPerform a thoracentesis a thoracentesis DemonstrateDemonstrate a a

willingness to be critically willingness to be critically evaluated by peersevaluated by peers

Objectives typically:Objectives typically: KnowledgeKnowledge SkillSkill AttitudeAttitude

Introduction to Physical OceanographyIntroduction to Physical Oceanography- Course Goals- Course Goals

Compare and selectCompare and select sources of data useful for the study of sources of data useful for the study of important physical variables and particular processes. important physical variables and particular processes. What instruments are used for measuring each variable? What instruments are used for measuring each variable? What is their accuracy and limitations? What is their accuracy and limitations? What historic data exists? What historic data exists? What platforms are used? Satellites, ships, drifters, moorings. What platforms are used? Satellites, ships, drifters, moorings.

Analyze and describeAnalyze and describe the important physical processes in the the important physical processes in the ocean. ocean. What are the physical properties of sea water? What are the physical properties of sea water? What are the important thermodynamic and dynamic processes What are the important thermodynamic and dynamic processes

influencing the ocean? influencing the ocean? What equations describe the processes and how were they derived? What equations describe the processes and how were they derived? What approximations were used in the derivation? What approximations were used in the derivation? Do the equations have useful solutions? Do the equations have useful solutions? How well do the solutions describe the process? How well do the solutions describe the process? Which processes are poorly understood? Which are well understood? Which processes are poorly understood? Which are well understood?

DescribeDescribe the processes that govern the distribution of ocean the processes that govern the distribution of ocean currents, the interaction of the ocean with the atmosphere. currents, the interaction of the ocean with the atmosphere. Describe how the ocean can influence weather patterns, especially Describe how the ocean can influence weather patterns, especially

El Niño. El Niño. Describe how the ocean can influence climate and cause abrupt Describe how the ocean can influence climate and cause abrupt

climate change. climate change.

Course Goals - Physics 105 - How Things Work

• To begin to see science in everyday life • To learn that science isn't frightening • To learn to think logically in order to solve

problems • To develop and expand your physical

intuition • To learn how things work • To begin to understand that the universe is

predictable rather than magical • To obtain a perspective on the history of

science and technology

Knowledge ObjectivesKnowledge Objectives Three main Three main

knowledge domainsknowledge domains Recall of informationRecall of information Conceptual Conceptual

understandingunderstanding Problem solvingProblem solving

Want to avoid undue Want to avoid undue focus on only one of focus on only one of these (most these (most commonly recall)commonly recall)

Sherry Chen, Virtues of Knowledge

Skills ObjectivesSkills Objectives

Usually “pass/fail”Usually “pass/fail” Standards of Standards of

performance may performance may be hierarchicalbe hierarchical

Alternative is to Alternative is to segment skills as segment skills as “can perform,” “can perform,” “has observed,” or “has observed,” or “can interpret.”“can interpret.”

Standard of Standard of PerformancPerformancee

Example of Example of SkillsSkills

Well qualified Well qualified or very or very competentcompetent

Measurement ofMeasurement of

BPBPAssessment of JVPAssessment of JVP

Familiar with or Familiar with or competentcompetent

Interpretation ofInterpretation of

CXRCXRDirect ophthal- Direct ophthal- moscopy to detect moscopy to detect hypertensive hypertensive changeschanges

Awareness or Awareness or minimal minimal familiarityfamiliarity

Cardiac Cardiac catheterizationcatheterizationPericardiocentesisPericardiocentesis

Attitude ObjectivesAttitude Objectives More difficult to More difficult to

conceptualize and articulateconceptualize and articulate Less commonly usedLess commonly used Very importantVery important

What is student’s attitude What is student’s attitude toward self-directed learning?toward self-directed learning?

How do students develop How do students develop attitudes about geriatric attitudes about geriatric patients?patients?

How do students develop How do students develop attitudes about other health attitudes about other health care professionals?care professionals?

Stuart Davis, New York Elevated , 1931

Communicating Course Goals Communicating Course Goals and Objectivesand Objectives

SyllabusSyllabus Course websiteCourse website Introduction of Introduction of

coursecourse Letter to studentsLetter to students

Juan Gris, The Newspaper, 1916

Course Goals and Objectives – Course Goals and Objectives – Best PracticesBest Practices

3) Selecting Appropriate 3) Selecting Appropriate Teaching Teaching

and Learning Methodsand Learning Methods Paradigm shifts in last Paradigm shifts in last

3 decades:3 decades: From focus on From focus on

instruction instruction

-> focus on learning-> focus on learning From conveying facts -> From conveying facts ->

motivating student to motivating student to actively engage materialactively engage material

Requires balance Requires balance between instructor's between instructor's leadership and student leadership and student initiativeinitiative

Thomas Eakins, Portrait of Professor W.D. Marks, 1886

What Makes for Good TeachingMark Krichevsky, M.D.Larry Hanson, M.D.October 26, 2006

Selecting Appropriate Teaching Selecting Appropriate Teaching

and Learning Methodsand Learning Methods Methods need to reflect objectivesMethods need to reflect objectives

Example: breath soundsExample: breath sounds Knowledge of how breath sounds are Knowledge of how breath sounds are

generated and transmitted may be generated and transmitted may be addressed in lecture, reading materials, or addressed in lecture, reading materials, or small groupssmall groups

Skills differentiating normal and abnormal Skills differentiating normal and abnormal breath sounds can use guided audio breath sounds can use guided audio recordings in lecture or small groupsrecordings in lecture or small groups

Skill of listening to patient’s lungs with a Skill of listening to patient’s lungs with a stethoscope requires clinical instructionstethoscope requires clinical instruction

LecturesLectures

Pro’sPro’s FamiliarFamiliar Useful for Useful for

communicating communicating knowledgeknowledge

Arguably most efficient Arguably most efficient use of faculty timeuse of faculty time

Introduce/explain new Introduce/explain new materialmaterial

Provide pace and Provide pace and framework to courseframework to course

Integrate with other Integrate with other teaching methodsteaching methods

Con’sCon’s EffectivenessEffectiveness

Not individualized or self-Not individualized or self-directed by learnerdirected by learner

Retention less than active Retention less than active methodsmethods

Challenging re: skills, Challenging re: skills, attitudesattitudes

Limits dialog/cross-talkLimits dialog/cross-talk AcceptanceAcceptance

Issues for adult learners – Issues for adult learners – respect for autonomy, respect for autonomy, consideration of unique consideration of unique life experiences and life experiences and knowledge, relevance, knowledge, relevance, practicalitypracticality

Boredom/restlessnessBoredom/restlessness

Optimizing Lectures – Optimizing Lectures – Preparation/OrganizationPreparation/Organization

Most common error is trying to teach too Most common error is trying to teach too muchmuch Don’t try to work in everything Need to know Don’t try to work in everything Need to know

learning context (knowledge base of students, learning context (knowledge base of students, curriculum structure)curriculum structure)

Identify truly key elements and cover them Identify truly key elements and cover them wellwell

Help students see forests as well as treesHelp students see forests as well as trees Example: if lecture covers all fungal infections, which Example: if lecture covers all fungal infections, which

are most common? Most serious? are most common? Most serious?

Optimizing Lectures – Optimizing Lectures – PresentationPresentation

Organization (lecture/notes)Organization (lecture/notes) Enthusiasm, confidence, humorEnthusiasm, confidence, humor Interactivity, respectInteractivity, respect Goes beyond written materialsGoes beyond written materials Has sense of difficult areas and can Has sense of difficult areas and can

explain themexplain them Use of aids, demonstrationsUse of aids, demonstrations Addresses relevanceAddresses relevance Expert subject knowledgeExpert subject knowledge

Optimizing Lectures – Optimizing Lectures – PresentationPresentation

Organization (lecture/notes)Organization (lecture/notes) Enthusiasm, confidence, Enthusiasm, confidence, humorhumor Interactivity, respectInteractivity, respect Goes beyond written materialsGoes beyond written materials Has sense of difficult areas and can Has sense of difficult areas and can

explain themexplain them Use of aids, demonstrationsUse of aids, demonstrations Addresses relevanceAddresses relevance Expert subject knowledgeExpert subject knowledge

Small Group ActivitiesSmall Group Activities

Many different Many different definitions and definitions and formatsformats

Main featuresMain features Active ParticipationActive Participation Face-to-face Face-to-face

contactcontact Purposeful activityPurposeful activity

Sylvia Sleigh, A.I.R. Group Portrait, 1978

Active/Interactive Learning Active/Interactive Learning TechniquesTechniques

Pro’sPro’s Improved retentionImproved retention Improved application of Improved application of

knowledgeknowledge Improved Improved

motivation/acceptance/motivation/acceptance/attendanceattendance

Acquisition of enduring Acquisition of enduring skills (incl. speaking, skills (incl. speaking, reasoning, negotiation, reasoning, negotiation, cooperation, leadership)cooperation, leadership)

Easier to discuss/model Easier to discuss/model attitudesattitudes

Facilitates faculty and Facilitates faculty and peer mentorship of peer mentorship of studentsstudents

Con’sCon’s Less familiar formatLess familiar format Requires significant time Requires significant time

to plan and start up to plan and start up (logistics, stimulus (logistics, stimulus material)material)

Require additional faculty Require additional faculty time (small groups must time (small groups must be small enough for all to be small enough for all to participate: 5-8 students)participate: 5-8 students)

All groups are not equalAll groups are not equal All members of a given All members of a given

group are not equalgroup are not equal Assessment/grading Assessment/grading

issues less issues less straightforward straightforward

Problem Based Learning

Rusty Kallenberg, M.D.November 2, 2006

Small Group FacilitationCharlie Goldberg, M.D., Shawn Harrity, M.D.

March 8, 2007

Teaching Practical/Clinical Teaching Practical/Clinical SkillsSkills

Active participation of Active participation of learnerlearner

Setting a good exampleSetting a good example Integration of clinical Integration of clinical

and basic science as and basic science as possiblepossible

Adequate opportunities Adequate opportunities for practicefor practice

Close observation and Close observation and feedbackfeedback

Rembrandt, The Anatomy Lecture of Dr. Nicolaes Tulp, 1632

Teaching/Learning Methods – Teaching/Learning Methods – Best PracticesBest Practices

4) Assessment of Students – 4) Assessment of Students – UCSD PoliciesUCSD Policies

503.Grading Policy - School of Medicine503.Grading Policy - School of Medicine [En 10/29/85; Rt by Assembly 3/4/86] [En 10/29/85; Rt by Assembly 3/4/86] 

(A)(A) With the exception of courses offered in the With the exception of courses offered in the first quarter, which will be graded P (pass); F first quarter, which will be graded P (pass); F (fail), the work of all students in those (fail), the work of all students in those courses designated by the Faculty of the courses designated by the Faculty of the School of Medicine as constituting the Core School of Medicine as constituting the Core Curriculum and the fourth year clinical Curriculum and the fourth year clinical clerkships will be reported in terms of three clerkships will be reported in terms of three grades: H (honors); P (pass); F (fail). [Am grades: H (honors); P (pass); F (fail). [Am 4/25/95; Am 1/29/02] 4/25/95; Am 1/29/02] 

(B)(B) In all other courses, grades shall be reported In all other courses, grades shall be reported as either S (satisfactory) or U as either S (satisfactory) or U (unsatisfactory). (unsatisfactory). 

(C)(C) No grade-points per unit shall be assigned. No grade-points per unit shall be assigned. 

Assessment of Students – Assessment of Students – LCME PoliciesLCME Policies

Criterion-Referenced vs. Norm-Criterion-Referenced vs. Norm-Referenced GradingReferenced Grading

Criterion-Referenced vs. Norm-Criterion-Referenced vs. Norm-Referenced GradingReferenced Grading

Criterion-Referenced Criterion-Referenced (absolute standard) (absolute standard) Students not compared Students not compared

to each other. to each other. Based on standards of Based on standards of

competence or content competence or content masterymastery

Necessitates Necessitates determination of the determination of the standard standard beforebefore the the examexam

Helpful to use items of Helpful to use items of known importance or known importance or difficultydifficulty

Example: Those scoring Example: Those scoring <60% fail<60% fail

Norm-Referenced Norm-Referenced (relative standard)(relative standard) Grades determined by Grades determined by

looking at performance looking at performance of of allall students students afterafter the the examexam

Examples: bottom 10% Examples: bottom 10% of class fail; scoring of class fail; scoring <1.2 SD below mean <1.2 SD below mean fail fail

Criterion-Referenced vs. Norm-Criterion-Referenced vs. Norm-Referenced GradingReferenced Grading

Criterion-Referenced Criterion-Referenced (absolute standard) (absolute standard) Students not compared Students not compared

to each other. to each other. Based on standards of Based on standards of

competence or content competence or content masterymastery

Necessitates Necessitates determination of the determination of the standard before the standard before the examexam

Helpful to use items of Helpful to use items of known importance or known importance or difficultydifficulty

Example: Those scoring Example: Those scoring <60% fail<60% fail

Generally preferredGenerally preferred

Norm-Referenced Norm-Referenced (relative standard)(relative standard) Grades determined by Grades determined by

looking at performance looking at performance of of allall students after the students after the examexam

Examples: bottom 10% Examples: bottom 10% of class fail; scoring of class fail; scoring <1.2 SD below mean <1.2 SD below mean fail fail

Formative vs. Summative Formative vs. Summative AssessmentsAssessments

FormativeFormative Used to provide Used to provide

data to learner on data to learner on interim progress interim progress and give feedback and give feedback that permits that permits additional additional improvementimprovement

Primarily a learning Primarily a learning experienceexperience

SummativeSummative Assessment at the Assessment at the end of a course or end of a course or unit that indicates unit that indicates the degree of the degree of proficiency proficiency achieved. achieved.

Primarily an Primarily an evaluative evaluative experienceexperience

Uses and Limitations of Various Uses and Limitations of Various Test Formats - MCQTest Formats - MCQ

Goal: you get the Goal: you get the question right because question right because you know the answer, you know the answer, you miss it if you you miss it if you don’t.don’t.

Advantages: computer Advantages: computer scoring, statisticsscoring, statistics

Disadvantages: Easier Disadvantages: Easier to test factual recall to test factual recall than higher level than higher level operations; can’t test operations; can’t test skills or attitudes wellskills or attitudes well

Johannes Vermeer, A Lady Writing, c. 1665

Case, SM, Swanson, DB. Constructing Written Test Questions for the Basic and Clinical Sciences, 3rd Edition. Philadelphia: National Board of Medical Examiners, 1998.

http://www.nbme.org/about/itemwriting.asp

Uses and Limitations of Various Uses and Limitations of Various Test Formats - MCQTest Formats - MCQ

Write to choose Write to choose best answer (e.g., best answer (e.g., Case vignette with Case vignette with stem ‘what is the stem ‘what is the most likely most likely diagnosis?”)diagnosis?”)

Avoid multiple Avoid multiple true/false, or true/false, or ‘negative’ ‘negative’ questionsquestions

Gerrit Dou, Man Interrupted at His Writing, 1635

Uses and Limitations of Various Uses and Limitations of Various Test Formats - MCQTest Formats - MCQ

Avoid technical flaws:Avoid technical flaws: Grammatical cuesGrammatical cues Logical cluesLogical clues Absolute terms (always/never)Absolute terms (always/never) Longer or more specific correct answerLonger or more specific correct answer Avoid superfluous info or ‘trick Avoid superfluous info or ‘trick

questions’questions’ Avoid focus on minutiaAvoid focus on minutia

Non-abordal nittering usually Non-abordal nittering usually involves aninvolves an

DacklexDacklex GrumeldGrumeld TelloidTelloid UstibleUstible

Non-abordal nittering usually Non-abordal nittering usually involves ainvolves ann

DacklexDacklex GrumeldGrumeld TelloidTelloid UstibleUstible

Regrallification becomes Regrallification becomes necessary whennecessary when

The gudges noogleThe gudges noogle The rekob flanes trelsate, and the The rekob flanes trelsate, and the

vosts fail to floncevosts fail to flonce The breg fribblesThe breg fribbles The hooluphs elgageThe hooluphs elgage

Regrallification becomes Regrallification becomes necessary whennecessary when

The gudges noogleThe gudges noogle The rekob flanes trelsate, and the The rekob flanes trelsate, and the

vosts fail to floncevosts fail to flonce The breg fribblesThe breg fribbles The hooluphs elgageThe hooluphs elgage

Nuriles are trassed by Nuriles are trassed by yukorrhea becauseyukorrhea because

All their obblers are sushedAll their obblers are sushed Their fleepers are always tolloidableTheir fleepers are always tolloidable The ning-dephle is usually rungedThe ning-dephle is usually runged Their snarms are never grovidTheir snarms are never grovid

Nuriles are trassed by Nuriles are trassed by yukorrhea becauseyukorrhea because

All their obblers are sushedAll their obblers are sushed Their fleepers are always tolloidableTheir fleepers are always tolloidable The ning-dephle is usually rungedThe ning-dephle is usually runged Their snarms are never grovidTheir snarms are never grovid

The ustible can be used inThe ustible can be used in

SluntingSlunting FreepingFreeping NitteringNittering GrungingGrunging

The ustible can be used inThe ustible can be used in

SluntingSlunting FreepingFreeping NitteringNittering GrungingGrunging

Uses and Limitations of Various Uses and Limitations of Various Test Formats – Short AnswerTest Formats – Short Answer

Can be pencil/paper or Can be pencil/paper or electronic (superlist)electronic (superlist)

Eliminates cuingEliminates cuing Increased grading Increased grading

timetime Requires more Requires more

judgment by grader judgment by grader than MCQ format; than MCQ format; more risk for more risk for ambiguity.ambiguity.

Gerard Terborch, Officer Writing a Letter, 1671

Uses and Limitations of Various Uses and Limitations of Various Test Formats – EssayTest Formats – Essay

Can be open or close Can be open or close ended questionsended questions

Tests skills such as Tests skills such as writing, reasoning, and writing, reasoning, and arguing abilitiesarguing abilities

Objective grading Objective grading criteria are challengingcriteria are challenging

Labor intensive to Labor intensive to gradegrade

Generally avoided Generally avoided unless offers unique unless offers unique advantages over other advantages over other formatsformats

Carlo Dolci, St. Matthew Writing His Gospel, 1671

Uses and Limitations of Various Uses and Limitations of Various Test Formats – MiscellaneousTest Formats – Miscellaneous

Direct observation Direct observation with checklist or with checklist or rating formsrating forms Most useful for Most useful for

skills/attitudesskills/attitudes Self-assessmentSelf-assessment

More useful in More useful in formative than formative than summative summative assessmentassessment

Annunciation and Nativity (Altarpiece of Observation), 1470

Validity and Reliability Validity and Reliability IssuesIssues

Validity and Reliability Validity and Reliability IssuesIssues

ValidityValidity – – Does it Does it measure what it is measure what it is supposed to supposed to measure measure (objectives)? (objectives)? SubjectiveSubjective judgmentjudgment Helpful to solicit Helpful to solicit

input multiple peopleinput multiple people If validity in doubt, If validity in doubt,

hugehuge hot button hot button issue for studentsissue for students

ReliabilityReliability – – Does it Does it produce consistent produce consistent results?results? Reuse of banked Reuse of banked

test itemstest items Consistent scores Consistent scores

on different parts of on different parts of the testthe test

Assessment of Students – Assessment of Students – Policies for the SyllabusPolicies for the Syllabus

Evaluation should be linked to objectives, and use Evaluation should be linked to objectives, and use appropriate methodologiesappropriate methodologies

Criterion-referenced systems preferred if possibleCriterion-referenced systems preferred if possible Interim formative feedback helpful (and required Interim formative feedback helpful (and required

by LCME) – early and often evaluation has by LCME) – early and often evaluation has advantagesadvantages

Grading policy in syllabus should be clear, explicit Grading policy in syllabus should be clear, explicit and as simple as possibleand as simple as possible

Weighting should be in accordance with course Weighting should be in accordance with course objectivesobjectives

Avoid multiple ways to fail the course (e.g., many Avoid multiple ways to fail the course (e.g., many noncompensatory domains)noncompensatory domains)

Students need to have concerns heardStudents need to have concerns heard

Assessment – Best PracticesAssessment – Best Practices

5) Contact with Students5) Contact with Students Syllabus should Syllabus should

outline how and when outline how and when to contact you.to contact you.

Email, office hours, or Email, office hours, or both? Who decides?both? Who decides?

““Approachability” Approachability” intensively scrutinized intensively scrutinized by student before by student before approach is madeapproach is made

Role of “student Role of “student liaison committees”liaison committees”

Grievance proceduresGrievance proceduresPaul Gauguin, The Conversation, 1885

6) Textbooks6) Textbooks Selection of a text Selection of a text

should be a late, not should be a late, not early considerationearly consideration

Selection of text is Selection of text is subservient to course subservient to course objectivesobjectives

Is it to be read cover to Is it to be read cover to cover, or only to answer cover, or only to answer specific questions?specific questions?

Distinguish between Distinguish between required and required and supplemental readingssupplemental readings

““Just in time” vs. “Just Just in time” vs. “Just in case” philosophiesin case” philosophies

Vincent van Gogh, Gauguin’s Chair with Books and Candle, 1888.

Consideration in Choosing Consideration in Choosing TextbooksTextbooks

Accuracy and currency of content Accuracy and currency of content Coherence and clarity of content Coherence and clarity of content Level of difficulty and interest for students Level of difficulty and interest for students

(challenging but not inappropriately difficult) (challenging but not inappropriately difficult) Cost Cost

Choose the less expensive work if it is of Choose the less expensive work if it is of comparable qualitycomparable quality

Choose paperbacks rather than hardbacks Choose paperbacks rather than hardbacks Limit the total cost of books for your course Limit the total cost of books for your course

by placing some works on reserve in the by placing some works on reserve in the library library

Size (heavy large texts are hard to carry) Size (heavy large texts are hard to carry) Format and layout (ease of reading) Format and layout (ease of reading)

7) Handouts (Class Notes)7) Handouts (Class Notes)

Class notes are an Class notes are an expectation at most expectation at most schoolsschools

Wide spectrum of Wide spectrum of what is handed outwhat is handed out OutlineOutline ProseProse PowerPoint handoutsPowerPoint handouts Key journal articlesKey journal articles

François Bonvin, Still Life with Book, Papers and Inkwell, 1876

Students’ Complaints About Students’ Complaints About NotesNotes

Lecture didn’t follow them closely Lecture didn’t follow them closely enoughenough

Lecturer followed them too closely Lecturer followed them too closely (i.e. read them)(i.e. read them)

Figures were unreadable or too smallFigures were unreadable or too small Variation in style between lecturersVariation in style between lecturers

Tips on NotesTips on Notes Have notes ready on timeHave notes ready on time Number all pages Number all pages List “Key Concepts” and “Key Terms”List “Key Concepts” and “Key Terms” Use major and minor headingsUse major and minor headings Leave wide margins and ample space for Leave wide margins and ample space for

annotationsannotations Appropriately annotate key figures and Appropriately annotate key figures and

make them largemake them large Add problems (3-5, ideally case-based) at Add problems (3-5, ideally case-based) at

end of text end of text Keep current, and reference current Keep current, and reference current

events and recent advances events and recent advances

What About Copyright What About Copyright Issues? Issues?

OGC helpful in assessing riskOGC helpful in assessing risk ALWAYS better to seek permission when ALWAYS better to seek permission when

incorporating copyrighted portionsincorporating copyrighted portions

What is the character of the use? What is the character of the use? Nonprofit, educational > commercialNonprofit, educational > commercial

What is the nature of the work to be used? What is the nature of the work to be used? Wider latitude for published, factual worksWider latitude for published, factual works

How much of the work will you use? How much of the work will you use? The less close to the complete work, the betterThe less close to the complete work, the better

What effect would this use have on the market for What effect would this use have on the market for the original or for permissions if the use were the original or for permissions if the use were widespread? widespread? Better if limited access, and ifit doesn’t compete with Better if limited access, and ifit doesn’t compete with

original workoriginal work

8) Other Resources8) Other Resources

Compiling a “useful Compiling a “useful links” librarylinks” library

VideoVideo AnimationsAnimations Skills Skills

demonstrationsdemonstrations PodcastingPodcasting OtherOther

Paul Goodnight, Links and Lineage

9) Course Evaluation9) Course Evaluation Last step of the course life cycleLast step of the course life cycle Critical for improvementCritical for improvement What data to collect?What data to collect?

Student evaluationsStudent evaluations CourseCourse LecturerLecturer

Local vs. national benchmarks (GQ)Local vs. national benchmarks (GQ) Outcomes dataOutcomes data

USMLE examsUSMLE exams Residency successResidency success

OtherOther How to implementHow to implement

Sustaining improvement vs. substantive redesign?Sustaining improvement vs. substantive redesign? When to infuse new blood?When to infuse new blood?

Learner and Program EvaluationShawn Harrity, M.D.Charlie Goldberg, M.D.June 7, 2007

Final ThoughtsFinal Thoughts

Syllabus design cannot be separated from Syllabus design cannot be separated from course designcourse design

Reflection, annual redesign, and year-Reflection, annual redesign, and year-round development cycle pay major round development cycle pay major dividends in quality of coursedividends in quality of course

Role of syllabus (and policies therein) key Role of syllabus (and policies therein) key part of the “unwritten curriculum” – the part of the “unwritten curriculum” – the implicit messages continually conveyed to implicit messages continually conveyed to studentsstudents

Stuart Davis, Rapt at Rappaport's,1952

How to reach me:

jess-mandel@ucsd.eduW: 858.534.1378Office: MTF 162A