Exploration Projects within the Education Research Grants

Program (84.305A) and Special Education Research Grants Program

(84.324A)

Allen RubyNational Center for Education Research

Information for Applying

Requests for Applications

Letter of Intent

IES Grants.gov Application Submission Guide

Application Package

Requests for Applications (RFA)

• A separate RFA for each grant program (84.305A and 84.324A)

• Describes the requirements for an application

• Requests for Applications are available on: http://ies.ed.gov/funding

• To be informed about the release of future RFAs, sign up for the IES Newsflash: http://ies.ed.gov/newsflash/

Letter of Intent (LOI)• A short description of your intended application

– PI, institution, collaborators– Budget – rough estimate– Up to 1 page abstract describing the work

• Purpose– Used by program officers to discuss your idea– Used by IES Office of Standards and Review to plan for

peer review process– Not used in the peer review process – superseded by

your application

• Submitted on http://iesreview.ed.gov

Application Packages

• Contains the forms to be filled out and submitted as your application

• Available at www.grants.gov– Help: support@grants.gov or 1-800-518-4726

• For the June 23, 2011 application deadline, packages will be available starting April 21, 2011

• For the September 22, 2011 deadline, packages will be available starting July 21, 2011

• Packages are specific for grant program and deadline

IES Grants.gov Application Submission Guide

• Instructions for completing and submitting the application package

• Available on http://ies.ed.gov/funding

Key Dates

Application Deadline

Letter of Intent Due

iesreview.ed.gov

Application Package Available

www.grants.gov

Start Dates

6/23/11 4/21/11 4/21/11 3/1/12 to

9/1/12

9/22/11 7/21/11 7/21/11 7/1/12to

9/1/12

Education Research Grants Program (84.305A)

Special Education Research Grants Program (84.324A)

Grant Topics

• All applications to 84.305A and 84.324A must be directed to a specific topic

• Identify the appropriate topic for your work– Read the Topic sections of the Request for

Applications (RFA)– Discuss with the appropriate program officer

(listed in the RFA)

Education Research Topics (84.305A)

• Reading and Writing• Mathematics and Science Education• Cognition and Student Learning• Social and Behavioral Context for Academic Learning• Education Technology• Effective Teachers and Effective Teaching

• Improving Education Systems: Policies, Organization, Management, and Leadership

• Postsecondary and Adult Education• Early Learning Programs and Policies• English Learners

Special Education Research Topics (84.324A)• Early Intervention and Early Learning in Special Education• Reading, Writing, and Language Development• Mathematics and Science Education• Social and Behavioral Outcomes to Support Learning• Transition Outcomes for Special Education Secondary Students• Cognition and Student Learning in Special Education• Professional Development for Teachers and Related Service

Providers• Special Education Policy, Finance, and Systems• Autism Spectrum Disorders• Technology for Special Education• Families with Children with Disabilities

Grant Research Goals

• All applications to 84.305A and 84.324A must be directed to a specific research goal (1 of 5)

• The goal describes the type of research to be done

• As a result, every application is directed to a specific topic/goal combination– Note topic and goal on your application (e.g., at

top of abstract)

The 5 Research Goals• Exploration

– The focus of this webinar

• Development and Innovation– Develop a new or modify an existing intervention

• Efficacy and Replication – Evaluate an intervention under ideal conditions

• Scale-up Evaluation – Independently evaluate an intervention under routine conditions

• Measurement– Develop and/or validate a measure

The Exploration Goal

Before Development Work Begins...

Need to understand the problem:• Examine the underlying processes that

affect education outcomes• Look for malleable factors that might be

targets for interventions• Identify what distinguishes between

effective and less effective practices

Before Evaluation Begins...

Identify education interventions or components of them that appear promising due to their association with improved student outcomes before committing resources to evaluation

In a Nut Shell...Development work and some evaluations begin with Exploration Projects that examine underlying processes for the purpose of informing the development of new interventions, modifying existing interventions, or identifying promising interventions for evaluation

Exploration• Explore the association between malleable factors

and education outcomes– A malleable factor can be changed by the education

system be it a characteristic of students (e.g., skills, behaviors), teachers (e.g., credentials, practices) or school (e.g., climate, size) , or an education program or policy

• Underlying processes that enhance or inhibit learning• Aspects of a school, district, or community associated with

beneficial education outcomes• Education interventions associated with beneficial

education outcomes (e.g., professional development, curricula, policies)

• Explore factors that mediate or moderate the relationship between malleable factors and student outcomes

Education Outcomes are for StudentsPreschool• School readiness• Developmental outcomes for infants and toddlers with

disabilities

Kindergarten through Grade 12• Academic outcomes in reading, writing, math and science• Behaviors, interactions, and social skills that support learning in

school and successful transitions to post-school opportunities• High school graduation• Functional outcomes that improve educational results,

transitions to employment, independent living, and postsecondary education for students with disabilities

Education Outcomes (continued)

Postsecondary• Access, persistence, completion• Achievement in gateway math & science courses• Achievement in introductory composition courses

Adult Education• Reading, writing, and math for basic and secondary

education and English Language Learners

Exploration Projects Should • Identify underlying processes that enhance or

inhibit learning. These may contribute to the development or modification of interventions

• Identify education interventions that may deserve rigorous evaluations

• Identify factors that mediate or moderate the underlying processes or the interventions

• Generate hypothesis concerning causal relations between factors and outcomes

• Contribute to theories of action

Exploration Projects Should NOT

• Test the efficacy of education interventions

• Examine non-malleable factors

• Examine malleable factors outside the control of the school system

The Application’s Research Narrative

• Key part of your application• 4 Sections

– Significance

– Research Plan

– Personnel

– Resources

• Each section scored and an overall score given• Requirements vary by topic & research goal• 25 pages, single spaced

Significance Section• Describes the overall project

– The research question to be answered– The malleable factors, moderators, and

mediators to be examined

• Rationale for the work– Theoretical justification

• Logic Models, Change Models

– Empirical justification• Related work

– Practical justification • importance of the variables (malleable factors,

mediators, moderators, outcomes)

Significance Section

• How work will lead to useful next step– Development or modification of

interventions to address the identified malleable factors or underlying process to improve student outcomes

– Identification of interventions for more rigorous evaluation

• Overall importance

Significance – 2 Problems Often Seen in Applications

• Unclear theory of change– Why should the malleable factor be related to

improved outcomes– A well laid out theory of change makes clear

what is expected to happen and in what order– Easy for reviewers to understand research

plan – why measure certain outcomes – Graphic can be helpful – e.g. a logic or

change model

Significance – 2 Problem Areas• Unclear Description of Intervention

– What the intervention is• Many components and may be applied at

different times – how fit together – Graphic may help

– Intervention not shown to be strong enough to expect an impact• Informational• Ensure fidelity

– Overly focused on actions not content• Ex.: 20 hours of PD held over 10 weeks but no

detail on what is to be covered in the sessions

Research Plan• Describe the work you intend to do

– How you will answer your research question

• Make certain the Research Plan is aligned to Significance section– All research questions should have

justification in Significance.

• Step-by-step process– Timeline to show when everything will be

done

All Research Plans Should Include• Clear, concise hypotheses or research

questions• Well-specified relations between

hypotheses, measures, and independent & dependent variables

• Clear description of data sources• Detailed descriptions of data analysis

procedures

The Research Plan Should Describe• Setting• Population and sample

– Sampling plan: inclusion and exclusion criteria– Size (power issue) and attrition– External validity

• Measures– Outcomes

• Proximal and distal• Sensitivity

– Other measures – Reliability and validity

Work May Include

• Original data collection with appropriate statistical analyses

• Secondary data analysis of existing datasets

• Secondary data analysis complemented by primary data collection

• Meta-analyses designed to determine moderators or moderators of effects

Primary Data Collection

• Sampling strategy• Sample characteristics• Variables to be measured• Procedures for data collection• Procedures for coding data• Describe links to secondary data if

relevant

If observational data are collected• How inter-observer reliability

maintained• How data be coded• How will data be quantified to support

prediction of relation between what was observed and outcomes of interest

Secondary Data Use• Database(s)

– Sample characteristics– Variables to be used– Access to and permission to use the data– Linking if multiple datasets used

• Malleable factor• Outcome variables• Mediators & moderators

Exploration Work is Non-Causal

• Descriptive analysis• Statistical correlational analysis

– Can include more complex methods to address selection issues

• Mediation analysis

Meta-analysis

• Criteria for including or excluding studies, and rationale

• Search procedures used • Coding scheme & procedures for

extracting data from studies• Procedures for ensuring reliability of the

coding

Meta-analysis• Demonstrate that sufficient numbers of studies

are available to support meta-analysis• Demonstrate that relevant information is

reported frequently enough that a database can be constructed

• Clearly describe effect size statistics, associated weighting functions, procedures for handling outliers, and any other adjustments

Research Plan: Analysis

• Describe how analysis answers research questions• Show your models • Address clustering• Describe how missing data will be handled• Check for equivalency of groups at start of study• Track attrition and check for attrition bias • Describe sensitivity tests of assumptions• Describe analysis of qualitative data and links to

quantitative analysis

Personnel Section• Describe key personnel

– Link each person and their expertise to their role in project - show that every aspect of project has person with expertise to do it

• Methodologists: show expertise in particular method to be used

• Substantive person for all issues addressed

• Do not propose to identify and hire key people after grant awarded

• Project management skills

– Give time contribution for each - show that every aspect has enough time from expert

• Orient CVs same way – specific to project– 4 pages plus 1 page for other sources of support

Personnel Requirements• Publication record and projected publications from

this grant are considered• Developers should discuss past success getting

developed interventions evaluated• If previous IES grant, discuss results• Evaluations require attention to objectivity should a

developer or persons with financial interest be involved– Efficacy projects: address how objectivity maintained– Scale-Up: Independent evaluation: developer can provide

routine implementation support

Personnel Strategies for PI

• Senior Researcher– Show adequate time to be PI– Make credentials clear: not all reviewers may know

• Junior Researcher as PI– Show adequate expertise not only to do work but to

manage project– Reviewers may be more comfortable if you have

senior person(s) on project to turn to for advice

Resources

• Show the institutions involved have the capacity to support the work– Do not use university boilerplate

• Show that all organizations involved understand and agree to their roles– What will each institution, including schools,

contribute to the project– Show strong commitment of schools and districts– Have alternatives in case of attrition

Resources (continued)• Appendix C should back this up with

– Detailed Letters of Support from research institutions, States, districts, schools

• Data issues– Document permission to use and access to

confidential data (letters in Appendix C)– Show familiarity with data – show that it can be

used to do the proposed work– If merging datasets, show that it can be done

Appendices• Appendix A (15 page limit)

– Figures, charts, and tables– Examples of measures– 3 pages to address past reviewer comments or to

argue that a proposal is a new submission

• Appendix B (10 page limit)– Examples of materials used in an intervention or assessment

• Appendix C (no page limit)– Letters of agreement (districts, schools, data providers, other

partners, consultants)– Clearly state responsibilities of the writer

Budget and Budget Narrative• Provide a clear budget and budget narrative

for overall project and each sub-award• IES Grants.gov Application Submission Guide

describes budget categories• Check RFA for specific budget requirements

for Research Goals and Grant Programs• Ensure agreement among Research

Narrative, Budget, and Budget Narrative

Award Information– Secondary data analysis or meta-analysis:

• Typical $100,000 to $300,000 per year (direct and indirect)

• Maximum 2 years and $700,000

– Primary data collection and analysis (with or without a secondary data analysis):

• Typical: $100,000 to $400,000 per year • Maximum 4 years and $1,600,000

Grant Submission• Make sure your institution is registered on grants.gov• Complete your online forms and upload PDFs• Authorized representative completes the process• Submit by 4:30:00 EST on deadline – earlier is safer• If problems uploading

– Contact Help Line 1-800-518-4726 and get a case number

• You should receive four emails – Grants.gov: assigns you a number that starts with GRANT– Grants.gov: your application is validated or rejected due to

errors. If the latter, correct and resubmit until validated.– Dept. of Ed: retrieved your application from Grants.gov– Dept. of ED: assigns you a number that starts with R305 or

R324

Application Review)• Carried out by the IES Office of Standards and Review• Compliance screening for format requirements• Responsiveness screening to topic and goal requirements• Assigned to review panel

– 2-3 reviewers (substantive and methodology)– If scored high enough, application is reviewed by full panel

• Many panelists will be generalists to your topic

• There will an expert in every procedure you use– Overall score plus scores on Significance, Research Plan,

Personnel, and Resources

• Resubmissions encouraged: address comments

Peer Review Process Information

http://ies.ed.gov/director/sro/peer_review/index.asp

Notification

• All applicants will receive e-mail notification of the status of their application

• All applicants receive copies of reviewer comments

• If you are not granted an award the first time, plan on resubmitting and talk to your program officer

Final Reminders

• Start early• Read the Request for Applications• Talk with the program officer• Start the online submission process

early

ies.ed.gov

Allen Rubyallen.ruby@ed.gov