the grant writing and review process at nih joshua smyth professor of biobehavioral health and...
Post on 25-Dec-2015
220 Views
Preview:
TRANSCRIPT
The Grant Writing and Review Process at NIH
Joshua SmythProfessor of Biobehavioral Health and Medicine; Associate Director,
SSRI
Rhonda BeLueAssociate Professor of Health Policy and Administration
Jenae NeiderhiserProfessor of Psychology
Danielle DownsAssociate Professor of Kinesiology and Obstetrics & Gynecology
Thanks to:
Lori Francis, Associate Professor of Biobehavioral Health
Brittany Frost, Social Science Research Institute
NIH Organization NIH Funding Mechanisms The Grant Writing Process
Focus on the R01 The NIH Review Process
Overview of Review Meeting The Scoring Process A Penn State example
Workshop Evaluation
Workshop Outline
I. The NIH Department of Health and Human
Services National Institutes of Health
25 Awarding Institutes/Centers aka ICs e.g., NICHD, NIMH, NIDA, NIA, etc.
Center for Scientific Review Office of the Director
The NIH Extramural Team
A. Program
B. Grants Management
C. Review
A. The Program/Institute Staff
Program Administrator
Maintains knowledge of scientific area Attends study section meetings Makes funding recommendations Monitors scientific progress Identifies scientific area of importance Reports to senior staff Development of programs and initiatives
B. Grants Management
Interprets Federal regulations and policies
Assures compliance with Federal regulations and policies
Monitors financial aspects of projects Interprets regulations and policy
C. Review:Scientific Review Group (1st
Level)
Center for Scientific Review (CSR) or NIH Institute & Center (IC)
Scientific Review Group (SRG) Non-federal scientists with relevant
expertise Led by a Scientific Review Officer (SRO)
http://www.csr.nih.gov/Roster_proto/sectionI.asp
C. Review:Advisory Council or Board (2nd
Level)
The potential awarding IC performs the second level of review
NIH program staff examine applications for impact (formerly “priority”) scores, percentile rankings, & summary statements against the IC’s needs
Program staff provide grant funding plan to Advisory Council or Board
Advisory Council or Board advises the IC director Director makes final decision
NIH Grant Application Cycle
Allocates Funds
Investigator NIHInstitution
Submits Application
Peer Review
Council Review
Funding Decision
Initiates Research
Conducts Research
Grant Application: It’s a process, not an event
1. Communicate with Program Officer Introducing ideas, getting feedback, pre-review
2. Get your proposal to the right review committee
Review the rosters and talk to colleagues Effectively wording the abstract Make a written request
3. Seek feedback from colleagues and consultants on drafts of the grant (prepare ahead!)
4. Consider who is likely to review your grant (review the rosters) and make sure to know and cite their work when relevant
5. Recognize that funding on first submission is rare
Ks: NIH Career Development Awards (K01, K02, K05, K07, K08, K22 [K99/R00])
P01: Research Program Project Grant P30: Center Core Grants R01: NIH Research Project Grant Program R03: NIH Small Grant Program R13: NIH Support for Conferences and
Scientific Meetings (R13, U13) R15: NIH Academic Research Enhancement
Award (AREA)
II. NIH Grant Mechanisms
R21: NIH Exploratory/Developmental Research Grant Award
R34: NIH Clinical Trial Planning Grant T series: NRSA Training Grants (T32, T34, T35,
T90, etc.) U series: Research Project Cooperative
Agreement Diversity Supplements: Research
Supplements to Promote Diversity in Health-related Research
Roadmap: NIH Roadmap Initiatives (Director’s Pioneer Award; Director’s New Innovative Program)
NIH Grant Mechanisms (continued)
R01 (primary focus today)
Used to support a discrete, specified, circumscribed research project
NIH’s most commonly used grant program
Body of the grant (Research Strategy) is 12 pages (mostly single spaced)
Generally awarded for 3 to 5 years Up to $500,000/year without exceptions
needed Advance permission required for $500,000 or
more (direct costs) in any year
New and Early Stage Investigators:A Competitive Edge
New Investigator has not previously served as a PI for an R01; may have been an investigator or received other smaller, developmental or research training awards
Early Stage Investigator (ESI) is within 10 years of completing her/his terminal research degree, or completing medical residency
Grant writing is: A skill like any other… But not the same skill as article writing
Instead, more of a problem-based writing activity (theory and practice problem)
III. The Grant Writing Process
Start early, make a timeline and STICK TO IT Should allow time for serious pre-submission
review and subsequent revision Develop a relationship with project
officers It is not possible to overdo clarity Let your passion come through in your
proposal Take advantage of early stage and new
investigator opportunities
A few preliminary tips
Know what has been done Know what has been funded
NIH website RePORTER (formerly CRISP) Decide on the problem
Important enough to get funded but simple enough to explain as clean design
in 12 pp for the R01; less for some other mechanisms
Assemble team Complementary skills;
Seniority/competence/other by association; People you can count on
Getting ready to write
Communicate with program officer Establish a relationship and trust
(funding decision) Acquire information on mechanism
and priorities Obtain input on aims/proposal
Getting ready to write
Face Page Table of Contents Performance Sites Other information
Project Summary (Description) Public Health Relevance Statement Facilities & Resources
Main Sections of the NIH Application(see Francis et al. for some examples)
Key Personnel Biosketches -- with personal
statements Budgets (for each study year)
Budget Justification Other sections (not discussed
today); for example: Clinical Trial and Human Embryonic
Stem Cell (HESC) List of Research Plan Attachments
More Sections
[Introduction – revisions only] Specific Aims: The basis for the proposal’s
organization Research Strategy
Significance and Innovation Approach
Preliminary studies Design Sample/recruitment/power analyses Procedures & measures Analyses
Main Sections of the R01: Specific Research Plan
Page Limit Guide: Plan your proposal with these limits in mind
Section of Application Page Limits
Introduction (for resubmission application only) 1
Specific Aims 1
Research Approach: R03, R13/U13, R21, R36, R41, R43, Fellowships (F), SC2, SC3
6
Research Approach: R01, single project U01, R10, R15, R18, U18, R33, R24, R34, U34, R42, R44, DP3, G08, G11, G13, UH2, UH3, SC1
12
Biographical Sketch 4
Page limits may vary for other funding mechanisms.
Check Funding Opportunity Announcement: http://grants.nih.gov/grants/guide/search_results.htm?scope=pa&year=active
Protection of Human Subjects Women and Minorities Planned Enrollment Table Children
References Cited Letters of Support Resource Sharing Plan Checklist SO START EARLY!
And More Sections
Clarity is key – each point follows naturally Tell a (scientific) story
Everything that needs to be there, is; nothing extra
Communicate your excitement Get feedback early on
And often
General themes of success
Impact and Significance Practice (2-3 sentences)
Prevalence of problem in population Important social concern/public health
problem Theory (model)
Building, testing, using Innovation
New directions, value added Compelling Preliminary Research
General themes to R01 Success
Specific Aims may be only part read by some reviewers No more than one page Tell the entire story
Functions as an abstract would in a manuscript
End with (actual) specific aims Hypotheses and aims must align
Keys to Success – Specific Aims
Background and significance Used to be an explicit section; no
longer This is where you ‘hook’ the reader
on your story Functions as the ‘introduction’ does in
a manuscript Likely no more than 2-3 pages
Keys to Success – Background
Research team collaboration no longer required to be documented Still very helpful
Preliminary study ‘types’ – document: Relevant previous research Research that demonstrates
competence in requisite domains Capacity to recruit in specific
populations and/or contexts Pilot data specific to proposal
Preliminary Studies
Methods are very important Overall -- clarity and detail May include a table that traces aims to
hypotheses to constructs to measures (table/s)
Is the design feasible? Are there gaps in the methods (e.g., fidelity
for interventions) Statistics are essential – product must match
aims Consider a methodologist team member
Include a detailed timeline
Keys to Success - Methods
Preference for representative samples Students only if relevant to
age/situation (e.g., college drinking) Generalizability from a single
entity (university, clinic, state) Justified exclusions
Unit of assignment is unit of analysis
Sample size/power analysis For each outcome/planned test
And More Methods: The Sample
Direct Costs Senior Personnel (PI, co-Is, project director)
(PSU fringe at 36.5%) Other Personnel (full time staff, RAs, part time
wages) (PSU fringe 36.5% for full time staff, 13.2% for Grad Assts AY; 7.9% for part time wages and summer)
Equipment Travel Participant/Trainee Support Costs
Budgets
Other Materials and supplies Publication costs Consultant services
Subawards/Consortium/Contractual Fees Other
Indirect Costs (~50% at PSU, but does not include all expenses)
Budget justification
Budgets, cont.
Receiving the Summary Statements: The Hardest Part!
1. Reviews critical, even harsh2. Reviewers usually find grant’s
weaknesses, while recognizing strengths
3. Summary statements spend much more time on critique than praise
4. Many investigators experience a mixture of rage and depression when they read their summary statements and easily lose perspective
5. Take a day or two (or more!) and then read again with a cooler head
Receiving the Summary Statements: Bouncing Back!
1. Ask experienced colleagues to read reviews
2. Don’t interpret criticism as hopeless3. Program Officer may be helpful in
clarifying critique4. If “discussed” (rather than triaged), you
have a chance of funding in next round5. The lower the initial score, the fewer
problems and more likely to be successful after revision
1. Persistence pays off in the grant process!!
2. Second submission must respond to the critiques through revision or clearly defending reasoning
3. Same reviewers may or may not review resubmission, but will see critique
Resubmission:Resilience and Flexibility!
Most Common Reasons for a Poor Score (in priority order)
Lack of impact or significance Lack of new or original ideas Hypotheses ill-defined, superficial, lacking, unfocused, or
unsupported by preliminary data Methods unsuitable, not feasible, not rigorous or not likely to
yield results; methods don’t clearly link to aims Design not logical, inappropriate instrumentation, poor
timing or conditions; doesn’t link well to aims Data management and analysis vague, not rigorous;
analyses don’t clearly link to aims Inadequate expertise or knowledge of field for PI; too little
time to devote to the work Poor resources or facilities; limited access to appropriate
population
When to Revise Basic idea was significant and
innovative or these can be bolstered
Design/measurement/analysis problems can be clarified (more information) or fixed
Need preliminary data Problem is poor writing
A. The Review Meeting B. Review Discussion C. The Scoring Process D. A Penn State Example
IV. The NIH Review Process
A. The Review Meeting: The SRO’s Role Prior to Meeting Point of contact until review group meets
(then project officer) Analyze submissions for completeness and
conflicts Recruit ad hoc reviewers as needed Schedule 1-2 day meeting Assign applications to reviewers (at least 3)
Primary, secondary, discussant Create review order based on preliminary
impact scores from best to worst within categories
Reviewers’ Role Prior to Meeting Familiarize self with criteria, mechanisms, and
scoring Review assigned applications
Assign scores to each criteria and other areas Write bulleted strengths and weaknesses for each
criteria Reviews are advice to institutes for funding
decisions, not advice to PI Post scores and comments on NIH Commons Read other reviews of assigned applications Prepare presentation of reviews Skim/read non-assigned applications
Format of the Review Meeting
SRO opening remarks Chair orientation New investigator R01 grants Other R01 grants Other grant types (R03, R15, R21,
R34) Applications discussed in order of
Impact Score; bottom 50% are not discussed
Confidentiality Review order Proposals below median within
each category may not be discussed
SRO Opening Remarks
Chair Orientation Start with reviewer impact scores
May differ from posted scores Goal of discussion is to clarify not
reach agreement If scores are similar, shorter discussion If scores are dissimilar, longer
discussion Recommended time
Primary – 5 minutes Secondary – 3 minutes Discussant – 2 minutes
B. Review Discussion Identify proposal Members in conflict leave Reviewers provide preliminary impact scores Reviews
Impact, Significance, Investigators, Innovation, Approach, Environment
Stress main points, do not repeat previous points Non-reviewers typically ask questions to clarify
Human Subjects issues affecting scoring Open discussion to entire committee
Review Discussion (continued)
Ask for reviewers impact scores again Identify the reviewers’ recommended
range Ask if anyone wants to score outside the
range Entire committee records impact score Discuss budget and other issues
C. The Scoring Process
1. Overall Impact Score: likelihood project will “exert a sustained, powerful influence on the research field(s) involved (1-9 scale)
2. A separate 1-9 score for each of 5 core criteria (Significance, Investigators, Innovation, Approach, Environment)
3. Additional review criteria help determine scientific and technical merit BUT are not scored separately
4. Additional review considerations are addressed by reviewers, but are not scored & are discussed after group scores
Score Criteria
Overall Impact: will project exert a sustained, powerful influence on the research field(s) as indexed by 5 core review criteria1. Significance: important problem addressed; how will this improve scientific knowledge, technical capability, and/or clinical practice
Impact Addresses: Probability of whether the research will exert a
sustained, powerful influence on the research field
Significance Addresses: Does the project address an important
problem or a critical barrier to progress in the field?
If the aims are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved?
What is the Difference Between Impact and Significance ?
Score Criteria (continued) 2. Investigators: PI & other researchers well suited to the project; appropriate experience & training; ongoing record of accomplishments; complementary & integrated experience; leadership approach, governance, and organizational structure appropriate for project
3. Innovation: the work challenges and seeks to shift current research or practice paradigms; utilizing novel theory, approaches or methods, instrumentation, or interventions; the work is novel
(Be innovative, but maybe not too innovative…)
Score Criteria (continued)4. Approach: strategy, methodology, analyses are well-reasoned and appropriate; potential problems & alternative strategies thought through; benchmarks set; risk is managed
Most common reviewer complaint is lack of detail here
Typically the longest section
5. Environment: the environment will contribute to the project’s success; institutional support, equipment, & other resources sufficient; unique features of the environment, subject population, collaborative arrangements
Additional Review Criteria (not scored)
Human Subjects:1. Protection of human subjects2. Data safety monitoring plan (clinical trials only)3. Inclusion of women, minorities, children4. Vertebrate animals5. Biohazards
Additional Review Considerations
1. Budget and period of support2. Select agent research (infectious
agents)3. Applications from foreign organizations4. Resource sharing plans
CRITERIA SCORING SYSTEMHIGH1. Exceptional: Exceptionally strong with essentially no
weaknesses2. Outstanding: Extremely strong with negligible
weaknesses3. Excellent: Very strong with only some minor weaknessesMEDIUM4. Very Good: Strong but with numerous minor weaknesses5. Good: Strong but with at least one moderate weakness6. Satisfactory: Some strengths but also some moderate
weaknessesLow7. Fair: Some strengths but with at least one major
weakness8. Marginal: A few strengths and a few major weaknesses9. Poor: Very few strengths and numerous major
weaknesses
CRITERIA SCORING SYSTEM (continued)
Minor Weakness: An easily addressable weakness that does not substantially lessen impact
Moderate Weakness: A weakness that lessens impact
Major Weakness: A weakness that severely limits impact
CRITERIA SCORING SYSTEM (continued)
Final Overall Impact Score:Mean of all reviewers’ final impact scores X 10Range = 10 (high impact) -- 90 (low impact)
NOTE: Scoring likely to produce applications with identical scores (“ties”). Thus, other factors (e.g., mission relevance, portfolio balance) will be considered when all other things are essentially equal
Research Plan Components Specific Aims Research Strategy
Includes Background & Significance; Preliminary Studies/Progress Report; Research Design & Methods
Facilities and Equipment Reflects the Environment criterion
For ESIs should describe the institutional investment in the success of the investigator
Biographical Sketch [NEW! as of Jan 25th, 2015] Personal statement – why well-suited for project, 4 pubs Contribution to science - you describe up to 5 of your
most significant scientific contributions (<=1/2 page each); up to 4 pubs or other products for each contribution area
Key Sections in R01 Proposal Format
D. A Penn State Example
Mock NIH Study SectionChair: Joshua Smyth, Associate Director, SSRIReviewer #1: Rhonda BeLue, Associate Professor of Health Policy and AdministrationReviewer #2: Jenae Neiderhiser, Professor of PsychologyReviewer #3: Danielle Downs, Associate Professor of Kinesiology and Obstetrics & Gynecology Review group members: Workshop attendees
Enhancing Peer Review Criteria: http://grants.nih.gov/grants/guide/notice-files/not-od-09-025.html
Page Limits: http://grants.nih.gov/grants/forms_page_limits.htm
Human Subjects: http://grants.nih.gov/grants/policy/hs/index.htm
SF424 guidelines for submission: http://grants.nih.gov/grants/funding/424/index.htm
Glossary: http://grants.nih.gov/grants/glossary.htm
Links of Interest
NIH RePorter (formerly CRISP)https://libraries.ucsd.edu/info/resources/nih-reporter-formerly-crispRevised Applicationshttp://grants.nih.gov/grants/policy/amendedapps.htmNIH Grant Writing Tip Sheets http://grants.nih.gov/grants/grant_tips.htm Getting an RO1http://sciencecareers.sciencemag.org/career_development/previous_issues/articles/1190/getting_an_nih_r01
General Resources
NSF Proposal Writinghttp://www.cs.cmu.edu/~sfinger/advice/advice.html
Other Proposal Writing Guides http://www.learnerassociates.net/proposal/
Reasons Proposals Failhttp://chronicle.com/article/How-to-Fail-in-Grant-Writing/125620/--let
More general resources
New and Early Stage Investigatorshttp://grants.nih.gov/grants/new_investigators/
New and Early Stage Investigator Resources
http://public.csr.nih.gov/Pages/default.aspx
http://grants.nih.gov/grants/oer.htm (forms, grant search, etc.)
NIH Websites
SSRI Listserv
New subscribers can join the SSRI listserv by sending mail to:
L-SSRI-subscribe-request@lists.psu.edu
top related