Medical Education:Medical Education:

An overview of the journal and field An overview of the journal and field of researchof research

Kevin W. EvaKevin W. EvaEditor-in-chiefEditor-in-chief

Program for Educational Research and DevelopmentProgram for Educational Research and DevelopmentDepartment of Clinical Epidemiology and BiostatisticsDepartment of Clinical Epidemiology and Biostatistics

McMaster UniversityMcMaster University

What is medical What is medical education research?education research?

•Difficult to defineDifficult to define

• Influence of students on Influence of students on patient carepatient care

• Assessment of Assessment of demands students demands students place on facultyplace on faculty

• War as an educational War as an educational experienceexperience

• Influence of legislative Influence of legislative changes in drug changes in drug promotion on medical promotion on medical studentsstudents

• Selection and Selection and recruitmentrecruitment

• Peer assessmentPeer assessment

• Delivering feedbackDelivering feedback

• Validity testing of Validity testing of assessment assessment instrumentsinstruments

• Student concerns Student concerns about peer physical about peer physical examinationexamination

Can’t be defined by Can’t be defined by contentcontent

Can’t be defined by Can’t be defined by contentcontent

The most consistent feature is variability

Med Educ Vol. 42(12): December 2008

• Cohort studyCohort study

• Correlational studyCorrelational study

• Psychometric Psychometric analysisanalysis

• Pre-post quasi-Pre-post quasi-experimental designexperimental design

• Grounded theoryGrounded theory

• EthnographyEthnography

• PhenomenologyPhenomenology

• Methodological Methodological suggestions suggestions

• Critical literature Critical literature reviewreview

Can’t be defined by Can’t be defined by methodsmethods

Can’t be defined by Can’t be defined by methodsmethods

The most consistent feature is variability

Advances in Health Sci Educ Vol. 12(2): May 2007

My own research My own research interestsinterests

Selection Instruction Assessment Expertise

Maintenance of

Competence

Self Regulation

What is medical What is medical education research?education research?

•Difficult to defineDifficult to define

•Science Science • "Systematized knowledge derived from "Systematized knowledge derived from

observation, study, and experimentation observation, study, and experimentation carried out in order to determine the carried out in order to determine the nature or principles of what is being nature or principles of what is being studied." studied." • ((Webster's New World Dictionary of the Webster's New World Dictionary of the

American Language)American Language)

What is medical What is medical education research?education research?

•Difficult to defineDifficult to define

•Science Science of medical education of medical education researchresearch• "Systematized knowledge derived from "Systematized knowledge derived from

observation, study, and experimentation observation, study, and experimentation carried out in order to determine the nature carried out in order to determine the nature or principles of or principles of issues relevant to the training issues relevant to the training and continuing education of health and continuing education of health professionalsprofessionals." ."

A tale of two emotionsA tale of two emotions

•Working in educational research Working in educational research provides:provides:

•Great joyGreat joy• Everyone has an opinionEveryone has an opinion

•Great frustrationGreat frustration• Everyone has an opinionEveryone has an opinion•

Goal of medical Goal of medical education researcheducation research

•To systematically gather To systematically gather empirical information empirical information pertaining to relevant issues pertaining to relevant issues to enable more sophisticated to enable more sophisticated opinions to be formedopinions to be formed

www.mededuc.com www.theclinicalteacher.com

““Publish or perish”Publish or perish”

• Survival in academia and advancement Survival in academia and advancement in training depend upon publicationin training depend upon publication• Possibly more than ever beforePossibly more than ever before

• Size matters!Size matters!• Impact factor; citations; h-indexImpact factor; citations; h-index

• Many in our field strive to publish Many in our field strive to publish without ever having been trained in without ever having been trained in resereseararch or publication strategiesch or publication strategies

What happens to a paper?What happens to a paper?

Submitted electronically

Checked by editorial office

Skim read by EiC

Sent to Deputy Editor

Sent to Reviewers (up to 4)

Reviews returned, recommendation made

Final decision made by EIC

Authors informed

Rejection

RejectionStatistical advice

•Some consequencesSome consequences

• Outright acceptance without revision Outright acceptance without revision extremelyextremely unusual unusual

• Time-frame given in which to undertake Time-frame given in which to undertake minor revision, reformat, or revise and minor revision, reformat, or revise and resubmitresubmit

• Revised scripts may be reviewed by EiC, or Revised scripts may be reviewed by EiC, or be sent out to reviewers (same be sent out to reviewers (same oror different) different)

• Appeals to EiC possibleAppeals to EiC possible• Will be forwarded to Quality and Standards Will be forwarded to Quality and Standards

Group if necessaryGroup if necessary

Decisions on manuscripts to Decisions on manuscripts to Medical Medical Education Education (2007)(2007)

•Total number = 1185Total number = 1185

• Reject without reviewReject without review 56%56%• Reject after reviewReject after review 23%23%• Accept after reviewAccept after review 21%21%

• Majority of acceptances offered only after revisions Majority of acceptances offered only after revisions are madeare made

What makes a good paper?What makes a good paper?

Discuss in small groupsDiscuss in small groups

Journal criteriaJournal criteria

Relevance Rigour

Originality Importance

Written well Ethically sound

Timely Interesting

Critical appraisal: Ask and answer at Critical appraisal: Ask and answer at least the following questions:least the following questions:

• Who was this written for?Who was this written for?

• What is the author trying to achieve & What is the author trying to achieve & why?why?

• What did they do? Why did they do it What did they do? Why did they do it that way?that way?

• What did they find?What did they find?

• So what?So what?

The review processThe review process

First pass by the EICFirst pass by the EIC• Is the paper in journal style?Is the paper in journal style?

• Read the Guidance for Authors!Read the Guidance for Authors!

• Is it relevant to the journal’s mission?Is it relevant to the journal’s mission?• The The ‘So what?’ ‘So what?’ factorfactor

•Has it been published before (literally Has it been published before (literally or figuratively)?or figuratively)?• If your paper has been rejected by If your paper has been rejected by

another journal, tell the EIC: Be honest & another journal, tell the EIC: Be honest & include comments from editor/reviewersinclude comments from editor/reviewers

How does the accompanying letter read?How does the accompanying letter read?

• Is it measured and balanced?Is it measured and balanced?• Does it ‘sell’ the paper convincingly?Does it ‘sell’ the paper convincingly?

• Is the paper well-written, proof-read, spell-checked Is the paper well-written, proof-read, spell-checked etc?etc?

• Is it interesting enough to read to the end?Is it interesting enough to read to the end?

• Is it the appropriate format/length?Is it the appropriate format/length?• Research Research paper? Short report? Letter? RGS? paper? Short report? Letter? RGS?

(TCT?)(TCT?)

The key question for The key question for reviewersreviewers

•Did you learn something?Did you learn something?• Good conceptual frameworkGood conceptual framework• Clear methodsClear methods• Appropriate analysisAppropriate analysis• Accurate discussionAccurate discussion

• Summarize findings and their relevanceSummarize findings and their relevance• Acknowledge limitations of method and Acknowledge limitations of method and

extent to which conclusions can be madeextent to which conclusions can be made

Reasons papers get rejectedReasons papers get rejected

Most commonly (this Most commonly (this year)year)

•Not a good fit for the journalNot a good fit for the journal

•Work doesn’t advance Work doesn’t advance understanding in the fieldunderstanding in the field• Writing trick: Do not mention the local Writing trick: Do not mention the local

context until the methods sectioncontext until the methods section

•Outcome measures are weak Outcome measures are weak (e.g., satisfaction ratings)(e.g., satisfaction ratings)

• The most important part of the paperThe most important part of the paper

• Should be catchy and descriptiveShould be catchy and descriptive

• Different styles for different journals:Different styles for different journals:• DeclamativeDeclamative• Independent variable, dependent variable, Independent variable, dependent variable,

interventionintervention• Pose/answer a questionPose/answer a question

Titles and AbstractsTitles and Abstracts

• Before submission, check the guidelines to Before submission, check the guidelines to authorsauthors

• Abstract/summaryAbstract/summary• TitleTitle• FormattingFormatting

• Double spacing, tables etc on separate Double spacing, tables etc on separate pages, referencing etcpages, referencing etc

• AcknowledgementsAcknowledgements• Conflicts of interestConflicts of interest• Ethics approvalEthics approval

Final thoughtFinal thought

The Journal’s main The Journal’s main responsibilities are towards its responsibilities are towards its

readers, readers, notnot the authors the authors

Medical EducationMedical Education

Editor in Chief: Dr Kevin Eva Editor in Chief: Dr Kevin Eva Monthly journal Monthly journal 2007 published 213 papers2007 published 213 papers(not including editorials and letters)(not including editorials and letters)

The Clinical Teacher The Clinical Teacher

Editor in Chief: Professor John SpencerEditor in Chief: Professor John SpencerQuarterly journal Quarterly journal 2007 published 64 papers2007 published 64 papers((not including editorials and letters)not including editorials and letters)

Contact: med@mededuc.com