Building Family Medicine Research in the US: Learning From Research Published in

2003

Donald Pathman, MD MPH, Anthony Viera, MDWarren Newton, MD MPH, UNC-Chapel Hill

For the

NAPCRG Committee on Building Research Capacity

AFMO Research Committee

NAPCRG WorkshopQuebecOctober 18, 2005

Workshop Objectives

Briefly describe national context regarding research capacity and the Published Research in Family Medicine Study—1999/2000

Summarize methods and preliminary results from the 2003 Study

Get suggestions for further analysis

Explore the meaning of the findings for the discipline

AFMO/NAPCRG Committee on Building Research Capacity

1990s concerns about number of family medicine researchers, quality of research and funding

Major strategic initiative to develop research capacity

Need to establish baseline and track progress: the Published Research in Family Medicine Study—1999/2000

Vital Signs of Family Medicine Research

How many articles?

How many in top-tier journals?

What topics?

Which journals?

Who are authors? Which institutions?

The Health of Family Medicine Research

Is Family Medicine research making progress?

How can Family Medicine improve its research productivity? Its impact?

Published Research in Family Medicine Study—1999/2000 Study Methods

Focus on empirical research, but no exclusions based on quality of study

Authors working in Family Medicine organizations

Painstaking, iterative search, with extensive cross-checking

1999/2000 Study Results

980 articles in 2 years, 70% with lead author FM researcher

35% in Family Medicine journals, with 236 total journals; 19 journals with > 10 articles

Authors with range of 1-28 articles over 2 years and median of 1 article

83% published by academic departments

2003 Follow-up:Current Project’s Goals

Find, quantify and describe U.S. family medicine’s research published in 2003

Use a process that is thorough, credible and reproducible

Widely disseminate findings

U.S. Family Medicine’s Research

• research published by US family medicine organizations and US family physicians

• output of the discipline’s organizations and members

• not . . .• all research in FM journals• all research relevant to FPs

Eligible Research Articles

Name at least 1 eligible US FM author

work in US FM organization (acad. dept; residency; AAFP. . .), or

any US family physician

Present research

present and analyze new data in a systematic way, or

undertake new analyses of existing data (e.g., meta-analyses, some other forms of systematic reviews)

Published in a professional journal, in English, with a 2003 publication date

Articles are eligible . . .

Regardless of the article’s: relevance to practicing FPs or discipline of FM topic (clinical, educational, HSR, . . .) research methods quality

Whether or not journal is: peer-reviewed indexed on Medline published in print or electronic form published in the US or international

Regardless of the FM authors’ academic degrees

Examples of ineligible articles

book chapters, articles in newsletters, unpublished reports

non-research scholarly published work editorials clinical reviews POEMs letters to the editor (unless they present new data or

analyses) case reports (case studies are included)

articles in FM journals not published by eligible FM author

research by non-FPs holding only adjunct appointments in FP organizations

Steps in Identifying 2003 Published Research

Hand search of 22 journals 302

NLM database searches (limits: 2003 pub date, English)

affiliation field search under terms 261“family” and “practice” or “medicine”

author field search of 385 authors of 792 or more eligible articles in 1999/2000,and sample of 75 authors w/ 1 article  

author field search of all 417 2003-4 U.S. 25NAPCRG members not previously searched

author field search of all identified 2003 79publishing authors not previously searched 

Review pub lists from 15 departments 19

765

22 Journals Searched by Hand

J Family Practice Family Medicine Academic Medicine J Am Board of Family Practice Medical Care JAMA Am J Public Health J American Geriatric Society Preventive Medicine Obstetrics & Gynecology J General Internal Medicine

J Rural Health Pediatrics Am J Preventive Medicine Arch Ped Adolescent Medicine South Medical J Teaching and Learning in Medicine Arch Internal Medicine Annals of Family Medicine Family Practice BMC Family Practice Am Family Physician

18 journals publishing > 10 elig. articles in 1999/2000

4 FP journals newly publishing research since 2000

Remaining Steps

Confirm eligibility/ineligibility of identified authors w/ dept. chairs, especially in joint departments

Validation/Integrity Steps

1. 2 reviewers assessed initial eligibility

2. Organization web sites, ABFM and AMA sites used to confirm author eligibility

3. a 3rd, “naive” reviewer confirmed “research eligibility” of 100% of 120

included articles “research ineligibility” of 100% of 107

excluded articles

4. Against “gold standard” of pub lists from 15 departments, 100 of 122 eligible articles were captured in our initial search steps (82%)

Number of Articles Identified

1. Total # research articles in 2003: 765 (55% ) 1999 & 2000 average 490

2. # research articles with FM- 458 (32% ) researcher as lead author

1999 & 2000 average 345

3. # research articles with FM- 307 (110% ) researchers as contributing authors

1999 & 2000 average 145

Number of FM Researcher-Authors Identified

4. Total # eligible FM researcher-authors 782in 2003

5. # eligible FM researchers who were 334 (43% of total)lead authors of at least one paper

Number of Articles perFM Researcher-Author

494

137

6628 25 9 8 3 4 1 2 1 0 2 1 1

0

100

200

300

400

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Mean – 1.89

Median – 1

Mode – 1

So Much from So Few

782 FM researchers authored 765 research articles

Top 15 (8+ articles each) authored 163 articles (21%)

Top 57 (5+ articles each) authored 398 articles (51%)

Number of Journals

282 total journals

161 journals published 1 article each

54 journals published 2 articles each

32 journals published 5 or more articles

each

Journals Publishing Most Articles

Family Med69 AJPH 10Academic Med 27 The J Family Practice 9J Am Board of Fam Med 26 Arch Peds Adoles Med 8Annals Family Med 20 Teach Learning in Med 8J Rural Health 17 Nicotine & Tobacco Res 8Southern Med J 12 Pediatrics 8J Gen Internal Med 12 Diabetes Care 7J Am Geriatrics Society 12 Medical Care 7J Women’s Health 12 J Palliative Med 7Am J Prev Med 11 Med Teacher 6JAMA 11 Arch Internal Med 6Preventive Med 11 J School Health 6J Am Osteo Assoc 11 Social Science & Med 6Am Family Physician 11

Examples of Journals Publishing 1 Eligible Article

Acad Emergency MedAIDSArch of DermatologyArthritis & RheumatismBlood Pressure MonitoringCancerChild Abuse & NeglectContraceptionDiabetes Technology & TherapeuticsTexas MedThe Aging MaleValue in HealthViolence and VictimsWilderness and Environ MedWomen’s Studies Quarterly

Family Medicine’s Journals

141 articles in FM journals (18% of all articles) 1999-2000 yearly average: 170 articles (35%)

U.S. JournalsFamily Medicine 69J Am Board of Family Medicine 26Annals of Family Medicine 20Am Family Physician 11

The Journal of Family Practice 9

InternationalFamily Practice 4BMC Family Practice 2

Articles in Top-Tier Med Journals

JAMA 11 10

Annals of Internal Med 4 1.5

BMJ 4 1.5

NEJM 3 2

Lancet 0 0

22 15

1999 -2000Average

2003

Article Breakdown *by Type of Organization

Academic FM Dept 675 88%

Residency (non-academic) 47 6%

Non FM organization 36 5%

Other FM organization 28 4%

Practicing FP 19 2%

Military site 7 <1%

* totals exceed number of articles, because some were co-authored by individuals in two types of organizations

Number of Researcher-Authors per Publishing Organization

Academic FM Departments

# authors # Depts % Depts 1-2 31 31% 3-6 35 35% >7 35 35%

FM Residencies in Non-University Hospitals

# authors # Residenicies % Residencies

1-2 24 77% 3-6 6 19% >7 1 3%

Stretch

Focused analyses

Preliminary analyses

Selected 4 issues relevant to policy for research in our discipline Funding sources Content International collaboration Role of PBRNs

Methods

Developed explicit coding rules

Random sample 250 articles

Practice coding 15 articles not in sample

2 coders independently coded hardcopies of sample articles with disagreements settled by consensus

Plan is to code every article in database

Issue 1: Funding for our research

Many believe our discipline poorly funded for research

Question of lack of support from federal government for family medicine research

How much funding do we receive? From what sources?

External to institution Federal government (NIH, AHRQ, HRSA,…) Professional organizations (AAFP, ADA,…) Foundations (non-discipline) (RWJ,…) Industry (pharma,…)

Own institution (University,…)

Funding sources

Funding sources

All noted sources of funding coded

Nothing about funding = “no funding source listed”

4

33

63

0 20 40 60 80 100

Owninstitution

only

No source

Externalsource(s)

% of articles 2003

Funding sources

n=250 random sample

11

13

15

16

30

39

0 20 40 60 80 100

Industry

HRSA

AHRQ

ProfessionalOrgs.

Foundations

NIH

% articles 2003

Most frequent sources

n=158 funded articles

Issue 2: Content

What are we publishing? 5 broad categories

Clinical Health services research Public health Educational Other

Detailed criteria for categories in coding rules

Content

3.2

3.2

18.4

35.6

39.6

0 20 40 60 80 100

Other

Public health

Educational

Clinical

HSR

% of articles 2003

n=250

Issue 3: International collaboration

Has been a goal of NAPCRG to promote international collaboration in research

How often family medicine researchers from other countries collaborate with U.S family medicine researchers on publications of research

Checked author information for any author affiliated with an organization in a country other than U.S.

Checked if US eligible co-authors listed as members of NAPCRG

International collaboration

12 out of 250 articles (4.8%)

5 international co-authors (on 4 articles) affiliated with family medicine organization

7 of the US eligible co-authors (on 5 articles) who collaborated with international co-author were members of NAPCRG

Issue 4: Role of PBRNs

Federation of Practice Based Research Networks (PBRNs), established 1997

Inventory updated 3/24/05 45 member PBRNs 9 U.S. affiliates 6 international affiliates

Practice based research networks

Scanned title, abstract, introduction and methods sections for information suggesting involvement of a PBRN

Looked not only for references to a network’s name but any indication that the study was carried out in an organized group of outpatient clinics or practices

Checked the compendium of FM PBRNs for names; scanned author names for known PBRN key players

Practice based research networks

5 of 250 (2%) sample articles were from a PBRN

Plan to continue with all articles in database

The future of this initiative

Finish polishing the data

Prepare summary report for journal publication.

Prepare additional manuscripts on selected topics (funding, top-producing depts., etc.)

Decide with Research Capacity Building Committee about periodically repeating this type of initiative in the future.

The Health of Family Medicine Research--2005

What do data from 2003 and growth since 2000 tell us?

Are we making progress?

How can we improve our productivity? Our impact?

Are changes needed in our discipline’s current approaches to promoting research?