review of biomedical research grants by the us

1 of 15

Department of Defense Congressionally Directed Medical Research Programs

U.S. Army Medical Research & Materiel Command

Congressionally Directed Medical Research Programs (CDMRP)

Presented to National Institutes of Health

Peer Review Advisory Committee

Presented byJanet Harris, Ph.D., R.N.

Colonel, US Army Nurse CorpsDirector

23 January 2006

US Army Medical Research and Materiel Command (USAMRMC)

2 of 15



CDMRP Philosophy

� Vision: Find and fund the best research to eradicate diseases and support the warfighter for the benefit of the American public

� Mission: We provide hope by promoting innovative research, recognizing untapped opportunities, creating partnerships, and guarding the public trust.

THE CENTRAL THEME IS INNOVATION

3 of 15



CDMRP History

� 1992: Grassroots advocacy heightened political awareness of breast cancer

� 1993: Congress appropriated $210M to the Department of Defense (DOD) budget for breast cancer research, managed by the CDMRP, after consultation with the Institute of Medicine (IOM) in fiscal year 1993 (FY93) and re-reviewed the program in FY97.

� 1996-2002: Additional research programs added:

� 1996: Neurofibromatosis

� 1997: Prostate Cancer and Ovarian Cancer

� 1999: Peer Reviewed Medical Research Program

� 2002: Prion, Tuberous Sclerosis, and Chronic Myelogenous Leukemia.

� These and other programs are managed by the CDMRP

4 of 15



� Funds added to the DOD budget by Congress [generally as DHP(RDT&E)]*

� Respond to targeted guidance from Congress

� Two-tier formal review of proposals – IOM Model

� Consumer advocate participation throughout process

� Vision is adapted yearly to facilitate rapid change and address research gaps

� Highly flexible management processes

CDMRP Unique Features

*Defense Health Program (Research Development, Test and Evaluation)

5 of 15



0

50

100

150

200

250

300

350

400

FY92 FY93-94

FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05

Institutionally BasedProgramsOsteoporosisResearch Program Defense Women'sHealth Res. Prog. National PrionResearch ProgramCooperative DOD/VAMedical Res. Prog.Tuberous SclerosisResearch ProgramMyeloproliferativeDisorders ResearchChronic MyelogenousLeukemia Res. Prog.Ovarian CancerResearch ProgramNeurofibromatosisResearch Program Peer ReviewedMedical Res. Prog.Prostate CancerResearch Program Breast CancerResearch Program

CDMRP Funding History

0

50

100

150

200

250

300

350

400

FY92 FY93-94

FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05

Institutionally BasedProgramsOsteoporosisResearch Program Defense Women'sHealth Res. Prog. National PrionResearch ProgramCooperative DOD/VAMedical Res. Prog.Tuberous SclerosisResearch ProgramMyeloproliferativeDisorders ResearchChronic MyelogenousLeukemia Res. Prog.Ovarian CancerResearch ProgramNeurofibromatosisResearch Program Peer ReviewedMedical Res. Prog.Prostate CancerResearch Program Breast CancerResearch Program

Awards 26 445 366 313 524 435 661 746 664 726 721 566 ~650

6 of 15



Training and Research Resources

Innovation-Based Research

ClinicalResearch

Traditional Research(e.g., R01-, P50-type awards)

FY06 Investment Strategy

TranslationalResearch

7 of 15



Program Cycle

VisionSetting

Release of ProgramAnnouncement

ProposalReceipt

Peer Review

ProgrammaticReview

Candidate Award List

CongressionalAppropriation

Receipt of

Funds

CGApproval Integration

Panel

� Science evaluation� Budget evaluation

� Programmatic relevance� Budget evaluation

8 of 15



Electronic Proposal/Grant Management

� CDMRP eReceipt System� Web-based proposal submission system

� Includes modules for Principal Investigators and Office of Sponsored Research Representatives

� Information collected in fields, text boxes, and PDF files

� Program and Peer Review Management Information System™ (P²RMIS™)

� Proprietary web-base system owned and used by Constella Health Sciences, our peer review contractor

� Electronic Grants System� Custom designed, state of the art database and business

system for the paperless management of medical research proposals and grants.

9 of 15



Advantages of Electronic Processes

� Reduced Applicant Cost: Uploading PDF files is less costly than copying and shipping proposal

� Reduced Program Cost: Reduction in labor during receipt and savings in reviewer shipping exceed cost of printing proposal copies

� Increased Communication: Reviewers, Chairperson, and Scientific Research Administrator can review critiques in advance of the meeting

� Time Savings: Compliance (administrative review), referral, and summary statement writing all streamlined through use of web-based tools

� Increased Efficiency: � Allows for collaboration between organizations by implementing

electronic workflows and virtual files

� Allows for real-time data transfer, multiple users, standardization of processes and unprecedented secure access to data

10 of 15



Standard Peer Review Scoring

Criteria Global Adjectival

10, 9 1.0 - 1.5 Outstanding

8,7 1.6 - 2.0 Excellent

6, 5 2.1 - 2.5 Very good

4,3 2.6 - 3.5 Good

2,1 3.6 - 5.0 Fair

11 of 15



Regular

(Idea Awards)Concept

Era of Hope

InnovatorClinical Trial Award

Referral Discipline-Focused Discipline-FocusedMechanism-Focused

Mechanism-Focused

Distribution Paper/CD-ROM P²RMIS™ (Web) Paper Paper/CD-ROM

CollectionCritique and Preliminary Scores

Brief Comment and Score

NothingCritique and Preliminary Scores

Pre-MeetingPanel-Wide Access to Comments

None NonePanel-Wide Access to Comments

Meeting Traditional Discussion NoneComparative Discussion

Dual Discussion: Proposal & Protocol

Global Score Numeric: 1.0 – 5.0 Adjectival Only: Outstanding - Fair

NoneSingle Numeric Global Score

Evaluation Criteria

10 – 1 NoneText Only: No Scoring

Separate Criteria for Proposal and Protocol

Summary Statements

Concatenated; Sent to Applicants

Sequential; Internal Use Only

Text Only: No Scoring

Separated Sections for Proposal and Protocol

Peer Review Processes

12 of 15



Correlations Between Final Global Scores and Final Criteria Scores for Idea Awards

Scores Correlated with Final Global Score

Pearson’s R Correlation

(Interval by Interval)

Research Strategy -0.886

Relevance -0.820

Innovation -0.876

Investigator -0.701

Environment -0.639

13 of 15



Lessons Learned

� Consumers are integral to peer review

� Efficiencies of cradle to grave electronic management

� Need to maintain dynamic flexible processes

� Tailor peer review processes to:

� Mechanism goals

� Time constraints

� Funding limitations

14 of 15



Ongoing Challenges

� Identifying Innovation

� Keeping Peer Reviewers engaged during panel discussions

� Congruency among narrative, criteria scores, and global score

� Optimizing panel size, reviewer workload, and length of panel discussion

� Meeting needs of multiple end users of critique

� Programmatic Review

� Applicant

15 of 15



Web Site – http://cdmrp.army.mil

review of biomedical research grants by the us

Documents