COVID-19 Recovery and Resilience Initiative (CRRI) RFP II

Proposals due: January 15, 2021

400 Main Street • E19-201 • Cambridge, MA 02142 • USA

COVID-19 RECOVERY AND RESILIENCE INITIATIVE (CRRI) REQUEST

FOR PROPOSALS (RFP)

PROPOSAL INSTRUCTIONS

FOCUS OF THE RFP

The COVID-19 Recovery and Resilience Initiative aims to catalyze research that has the potential to

answer the open policy questions presented by the COVID-19 pandemic and help decision makers

support their communities in the short and long term. J-PAL North America is now calling for

proposals from J-PAL affiliates, J-PAL post-doctoral fellows, or invited researchers. For this round of

funding, proposals are due by 5:00pm EST on Friday, January 15, 2021.

The COVID-19 Recovery and Resilience Initiative welcomes proposals in any sector that explore

potential strategies that support the recovery of low-income communities in the wake of the pandemic.

In particular, for this inaugural round additional funds are available for studies that focus on redressing

racial inequities in outcomes for youth and young adults. We are accepting applications for full research

projects, pilot studies, and travel/proposal development grants.

TABLE OF CONTENTS

Special Note: Covid-19 2

RFP Timeline 3

Eligibility Criteria 3

Proposal Types 3

Full Research Projects 3

Pilot Studies 4

Short-Term Research Management (STReaM) 4

Travel/proposal development grants 4

How to Apply for pilot or full funding 5

Required documents: 5

Additional Steps for Off-Cycle Proposals 7

CRRI RFP I

Proposals due: January 15, 2021

2

Additional Steps for Graduate Students 7

Important Notes 7

Guiding Questions for Applicants 8

Evaluation Criteria 9

Review Process 10

Appendix 1: Connection to J-PAL Mission 11

Appendix 2: Ethical Research and Risk Mitigation 12

Appendix 3: Grant Requirements 15

SPECIAL NOTE: COVID-19

Please note the following modifications to our request for proposal processes in response to the COVID-19 crisis. These are in place as of October 29, 2020 until noted otherwise.

1. All applications must follow J-PAL’s COVID-19 response guidelines in regard to

in-person interactions with subjects.

2. If your proposed project will feature in-person interactions with subjects, vendors, or partners,

please note that the evolving situation could result in changes to these guidelines at a future

date. Therefore, please be sure to include the following in your application

materials, which are listed in more detail in the “How to Apply” section:

● Budget Narrative: Include a discussion of what funds will and will not be spent in what

timeframes if the proposed project is paused, modified, or dissolved because of COVID-19

related disruptions.

● Proposal Narrative: Outline aspects of the proposed project that can be completed right

away vs aspects dependent on the status of the COVID response more broadly. The

proposal should also include plans for conducting fieldwork, and how this will evolve

should COVID-19 prevent in-person contact.

3. Per MIT guidance, J-PAL North America will not be sponsoring travel for the foreseeable

future. However, if you wish to include travel as a line item in your please address as indicated

in point 2 above.

The following funding opportunities are available in addition to regular proposals submitted to

this RFP:

● Proposals for NEW projects that will generate rapid, actionable insights that can help

inform policy responses to the COVID-19 pandemic. Proposals will be reviewed under our

standard evaluation review criteria as well as their potential to produce informative and

timely findings to address the current crisis.

CRRI RFP I

Proposals due: January 15, 2021

3

RFP TIMELINE CRRI RFP II Opens November 23, 2020

CRRI RFP II Closes January 15, 2021

Award Letters Released Week of March 8. 2021

ELIGIBILITY CRITERIA

J-PAL affiliates, J-PAL post-doctoral fellows, and researchers invited by J-PAL North

America (invited researchers) to participate in any North America Initiative are eligible to

apply for funding of any type. Applicants can submit a maximum of three proposals per 12-month

period to CRRI. PI and co-PI status are counted towards this limit.

Graduate students may apply for funding for full research projects, pilot studies, or travel/proposal development grants. Graduate students are not eligible to apply for Short Term Research Management Assistance (STReaM). To apply, graduate students must meet the following criteria.

● They have a J-PAL affiliate on their thesis committee (please note that having an invited researcher on a student’s thesis committee does not satisfy this criterion). The J-PAL affiliate does not need to be based at the same university as the graduate student.

o Pre-thesis PhD students who do not have an official thesis committee may also apply. Pre-thesis students are required to submit a formal letter of confirmation from an affiliate that explicitly states: “I am actively responsible for supervising this project/research and anticipate being on the student’s thesis committee.”

● They received a travel/proposal development grant from J-PAL for the specific project proposed or have documented evidence of successful piloting activities.

● For those graduate students who previously received funding from J-PAL, the total allowable per project budget is $50,000. For example, if a graduate student received a $5,000 travel/proposal development grant previously and subsequently applies for pilot funding for that same project, the maximum budget that can be requested is $45,000.

Graduate students may be awarded a maximum of two travel/proposal development grants and two grants for pilot/full study funding during their time as graduate students.

PROPOSAL TYPES

For additional details on what proposals should include, please see “Guiding Questions for Applicants.”

Full Research Projects

Full research projects are typically awarded $150,000 to $250,000, with a maximum of $400,000.

The award period may be up to three years. A full project proposal is one where applicants:

● Can propose a clear and well-developed research question

● Can provide detailed randomization design and power calculations

CRRI RFP I

Proposals due: January 15, 2021

4

● Can indicate outcomes of interest

● Can provide proof of commitment from partner organizations (in the form of letters of support) Full project proposals can be submitted for funding an ongoing study that has already begun without J-PAL North America funding. Pilot Studies

Pilot studies may be awarded a maximum of $50,000, with the exception of pilot projects that also

apply for J-PAL-funded STReaM support (please see below). The award period may be up to

three years. A pilot proposal is one where applicants:

● Seek to answer a particular research question; however, the design and

implementation require further testing and development before a full project

launches

● Can clearly explain how the pilot will lead to a randomized evaluation in the future,

although random assignment does not necessarily need to occur during the pilot

Projects that receive pilot funding may apply for additional funding in future RFPs.

Short-Term Research Management (STReaM)

Through this RFP, researchers can apply to receive short-term research management (STReaM). This is

a program that provides 3-6 months of research management support from J-PAL North

America staff. This support can be used for either full studies or pilots, and may include but is

not limited to: coordinating communication across stakeholders; refining randomization

implementation design and consent procedures; pilot design and implementation; and study process

monitoring. For more information about STReaM and to fill out an application, please see

the website or contact stream@povertyactionlab.org.

Pilot projects that apply for J-PAL-funded STReaM support may request a total budget of up to

$75,000, including the STReaM estimate as indicated in the budget template.

Travel/proposal development grants

Researchers may be awarded a maximum of $5,000 for one year. Travel/proposal development

grants are to be used for early stage activities with the goal of launching a randomized evaluation in

North America. Expenses covered under this award are intended for travel and other costs incurred

while doing fieldwork or building relationships with potential implementing partners. Please note

that travel/proposal development grants are paid by direct reimbursement to the PI

and cannot cover any activity that requires review by an IRB.

To apply, please complete and submit this form [LINK] to na_rfp@povertyactionlab.org

HOW TO APPLY FOR PILOT OR FULL FUNDING

CRRI RFP I

Proposals due: January 15, 2021

5

Required documents:

1. The proposal application cover sheet (available here [LINK]).

● Please save the coversheet as [PI Name]_[Topic Name].pdf

2. A 5-page narrative

● Please save the narrative as a Word document, titled [PI Name]_[Topic Name].doc(x)

● The narrative should address each of the points listed under “Guiding Questions for Applicants.” If necessary to fully address the guiding questions on Ethics and Risks, please include an addendum (details under 2A).

● The narrative may not exceed five pages in length. The minimum font size is 11 points, and margins may not be smaller than 1 inch. J-PAL North America will send back any proposals violating these formatting rules.

● The narrative should begin with an abstract of 150-200 words which J-PAL North America will post online if the project receives funding. The abstract should include information on the research question, the hypotheses, the intervention, the (potential) sample size, and the outcomes.

2A. Optional: A 1-page narrative addendum on ethics and risks

● If included, please save the addendum in the same document as the full narrative, using the same formatting conventions. Label the relevant section, “Addendum on Ethics and Risks.”

● The addendum may not exceed 1 page in length, for a total combined document of no more than six pages.

3. A complete budget (template available here [LINK]).

● Please save as a Word document, titled [PI Name]_[Topic Name].doc(x)

● Project budgets should reflect the actual needs of the project.

● All J-PAL North America Initiatives cap overhead charges at 9% of total direct costs.

● J-PAL North America does not cover PI effort (academic or summer months). We will support salary and fringe benefits for other project personnel.

● Enter the amount requested by line item in the budget template. o Institution/University costs are expenses incurred directly by the university/institution

receiving the award, such as a RA working at the PI’s university. o Partner costs are all costs not directly incurred by the university/institution, such as a

subaward to cover operational costs incurred by the implementing partner. o Travel costs should include a breakdown of how many trips are planned, the estimated

cost per trip, and the purpose of the trip.

o Any computer/equipment purchases should include an itemized list of purchases. For example, it should include how many laptops will be purchased and the purpose of the purchase.

● Budgeting for J-PAL’s courses (optional): o Staff training: J-PAL's staff training is designed for research staff working on randomized

evaluations conducted by our network of faculty affiliates, invited researchers, or our

CRRI RFP I

Proposals due: January 15, 2021

6

partner organizations. This five-day course provides technical guidance for designing, implementing, and analyzing data in randomized evaluations, with an optional focus on US-based studies that use administrative data. More details, including a sample course agenda, are available here.

o Executive Education: J-PAL’s flagship executive education course, Evaluating Social Programs, is designed for managers and researchers from NGOs, governments, and foundations. This five-day course provides participants with a thorough understanding of randomized evaluations and a pragmatic step-by-step guidance on conducting an evaluation. More details, including a sample course agenda, are available here.

● J-PAL will not approve costs labeled miscellaneous, contingency, or rent.

● J-PAL does not allow variance greater than 10% (by major line item) without prior approval. If modifications are needed, project staff are required to inform J-PAL and request approval in advance.

● If there is co-funding for the project, please complete the Total Project Budget and the J-PAL North America Budget tabs in the budget template.

● Expenses associated with the provision of detailed information on program costs are allowable.

4. A budget narrative

● Please provide justification for the expenses requested in your budget. Please save the budget narrative as [PI Name]_BudgetNarrative.doc(x)

5. Letter(s) of support from the partner organizations

● Full projects are required to provide a letter of support from implementation partners.

● Applicants for pilot funding are encouraged to submit letters of support.

● Letters of support should indicate willingness to share cost data, when appropriate.

6. Optional: Questionnaire for Short-Term Research Management (STReaM)

● Fill out the questionnaire here if you are interested in working with a research manager through STReaM.

Additional Steps for Off-Cycle Proposals

In rare cases, proposals facing time constraints due to factors outside their control may apply to the

RFP off-cycle. Off-cycle proposals will face the same scrutiny as proposals submitted during the RFP

round, and must include a justification for off-cycle submission. Please email your submission materials

to covid@povertyactionlab.org with “Off-cycle Proposal” in the subject line.

Additional Steps for Graduate Students

All proposals submitted by graduate students must include a letter of support from the J-PAL affiliate

on this student’s committee, in which the adviser attests to being on the student's thesis committee.

CRRI RFP I

Proposals due: January 15, 2021

7

Pilot or full proposals submitted by graduate students must also reference the student’s prior

travel/proposal development grant and /or document successful piloting activities.

Important Notes

● If a proposal is under review by a J-PAL North America initiative, it may not be

submitted to another J-PAL North America RFP unless explicitly given permission to do

so. This includes different versions of the same proposal.

● Applicants are encouraged to concurrently apply for approval from their respective Institutional Review Boards (Human Subjects Committees). The award of any J-PAL North America grant is contingent on approval from the host institution’s IRB as well as the IRB at MIT, known as the Committee On the Use of Humans as Experimental Subjects (COUHES).

● Applicants are encouraged to submit the application to their office of sponsored programs or contracts department as MIT will need official acceptance of the proposal and budget by your institution to process the subaward. You may do this after submitting a proposal but doing so before the award decision will help avoid delays.

● It is your responsibility to ensure the budget you submit is correct and follows your host institution’s policies for costs. If you wait until an award has been made by the RFP Review Board before getting approval from your planned host institution, you risk having an award that your institution cannot accept. We recommend that as soon as you submit your proposal to J-PAL North America (if not before), you send it through your host institution for their review and acceptance.

Send all documents to covid@povertyactionlab.org

Due: 5 pm EST on January 15, 2021

Questions? Email Caroline Garau cgarau@povertyactionlab.org

GUIDING QUESTIONS FOR APPLICANTS

A strong proposal will demonstrate academic rigor, an appreciation of potential risks, policy relevance, and alignment with J-PAL’s mission to reduce poverty by ensuring that policy is informed by scientific evidence. To enable reviewers to assess the merits of your proposal, please ensure that your narrative addresses each of the following questions. For additional details on the guidance provided to reviewers, please see “Evaluation Criteria.”

Relevance to public policy and issues of poverty

For more detailed guidance on related evaluation criteria, please see Appendix 1: Connection to J-PAL’s Mission

● How will answering the proposed research question and/or evaluating the proposed intervention generate benefits for people living in or at risk of falling into poverty?

● What pressing public policy questions in the United States will this study answer? What information will it provide to guide policy making in this area?

● How will lessons learned from this study have broader relevance or applications for policy or decision-making beyond this test case?

CRRI RFP I

Proposals due: January 15, 2021

8

Contribution

● How will this study contribute to advancing knowledge in the field?

● How does the proposed study compare with the existing body of research?

● What underlying economic theories have informed this study?

Value of research

● Why do you feel that the proposed study offers good “value for money” in terms of the insights it will provide?

Technical design

● How will the research design of this study enable you to effectively answer the questions outlined in your proposal?

● Are there threats that could compromise the validity of results? How do you plan to address these?

● What are the power calculations you are using as part of your research design?

o Please provide sufficient detail to help us understand your assumptions, and be sure to include:

▪ Parameters of your chosen average effect size (e.g. ITT, LATE, etc.) ▪ Outcome(s) on which your power calculations are based

▪ Units used in your power calculations

▪ The minimum detectable effect size

▪ Underlying assumptions about take-up

o We recognize that this may be more uncertain at the pilot stage. For pilot studies, please provide the same information based on your current thinking around what would constitute sufficient power if a randomized evaluation were eventually launched.

Ethics and risks J-PAL is committed to upholding high ethical standards, and to minimizing and mitigating risk to study

participants, staff, and the broader community impacted by research. These ethical standards and risks may include

but extend beyond what an IRB is designed to assess. Some proposals may have relatively brief responses to these

questions, while others may need to provide more details depending on the intervention or context.

J-PAL North America staff may be available to provide support for risk assessment or mitigation. For more detailed guidance on related evaluation criteria, please see Appendix 2: Ethical Research and Risk Mitigation.

● Please flag and describe any risks of unintended negative consequences for study participants, community members, staff, or others stemming from the intervention, study, or potential misuse or misinterpretation of study results.

● How do you plan to address these risks, if any?

● Why is the intervention being studied appropriate in this context for this population?

Viability of the project

CRRI RFP I

Proposals due: January 15, 2021

9

● What role will your chosen implementing partner play in this study? Why do you feel confident that they will be able to fulfill that role?

● What is your anticipated timeline for completing the study and analysis?

● Are there any logistical or political obstacles that might threaten the completion of the study, for example, government authorization or completion of multiple data use agreements?

Publishing data

For more detailed guidance on related evaluation criteria, please see Appendix 3: Grant Requirements

● If funded by J-PAL, do you plan to register the research project and publish data and replication code online when not precluded by privacy restrictions?

EVALUATION CRITERIA

These guiding questions are provided to all proposal reviewers, and discussed as part of the final decision-making

process.

Relevance to public

policy and issues of

poverty

● Does the proposal make the case for how answering the proposed research question and/or evaluating the proposed intervention has the potential to generate benefits for people living in or at risk of falling into poverty?

● Does the proposal make the case that the lessons learned from this study will have broader relevance or applications for policy or decision-making beyond this test case?

● Does the study address questions crucial to understanding pressing public policy issues in the United States?

● Is there demonstrated demand from policy makers for more/better information to influence their decisions in this area?

For more detailed guidance on these criteria and related examples, please see Appendix 1: Connection to J-PAL’s Mission

Contribution

● Does the study contribute to advancing knowledge in the field? Does it answer new questions, introduce novel methods, measures, or interventions?

● Is there academic relevance? How does the study compare with the existing body of research?

● Is the experiment clearly linked to underlying economic theories?

Value of research ● Is the cost of the study commensurate with the value of expected lessons learned?

Technical design

● Does the research design appropriately answer the questions outlined in the proposal?

● Power calculations and sample size

● For full studies: Does the proposal include power calculations based on a realistic and decision-relevant minimum detectable effect (MDE) size, accompanied by a clear explanation of the chosen average effect size parameters and underlying assumptions regarding

sample size and take-up rates?

● For pilot studies: Does the proposal include at least a “back-of-the-envelope” sense of the potential sample size and power to detect relevant impacts if a randomized evaluation were launched?

● Are there threats that could compromise the validity of results?

● If so, does the proposal sufficiently address those threats?

Ethics and Risks

● Are the risks of unintended negative consequences for study participants, community members, staff, or others stemming from the intervention, study, or potential misuse or misinterpretation of study results, low?

● Has the team taken appropriate steps to assess, monitor, and mitigate/prevent any identified risks?

● Has the team outlined a compelling argument for why the intervention is appropriate in this context for this population?

● Based on existing knowledge and research on related interventions or methods, is there reason to believe there may be risk of harm or other unintended negative consequences that the proposal did not identify?

For more detailed guidance on these criteria and related examples, please see Appendix 2: Ethical Research and Risk Mitigation.

CRRI RFP I

Proposals due: January 15, 2021

10

Viability of the project

● Is the relationship with the implementing partner(s) strong and likely to endure through the entire study?

● Are there any other logistical or political obstacles that might threaten the completion of the study, for example, government authorization or completion of multiple data use agreements?

● Is a realistic timeline for completing the study and analysis provided?

Publishing data

● Does the project advance best practices in research transparency?

● Is the researcher committed to registering the research project and publishing data and replication code online when not prec luded by privacy restrictions?

For more detailed guidance on these criteria please see Appendix 3: Grant Requirements

REVIEW PROCESS

Proposals are reviewed by two peer reviewers from J-PAL’s academic network. After peer review, the

CRRI RFP Review Board, typically consisting of Alicia Sasser Modestino, Larry Katz, and Judd Kessler,

reviews each proposal and makes all funding decisions. The Review Board may grant an unconditional

approval, a conditional approval (pending minor revisions), a request to revise and resubmit, or a

rejection. Regardless of funding status, all applicants to the RFP will receive redacted comments from

the referees.

In the event that an CRRI RFP Review Board member submits a proposal in the current round of the

RFP, they are required to recuse themselves from the Review Board. No spouse, partner, or

immediate family member of any individual named on a proposal may serve as a peer or Review Board

referee in the round in which the applicant’s proposal is under review.

If you would like to appeal a decision of the CRRI Review Board, please contact Caroline Garau

cgarau@povertyactionlab.org within one week of the decision. Please send a detailed write-up of the

reasons for this appeal (maximum two pages in length).

APPENDIX 1: CONNECTION TO J-PAL MISSION

It is important to the scientific directors and the staff that J-PAL NA uses our limited resources as

effectively as possible in line with our mission. One consideration in the RFP is the extent to which a

study aligns with J-PAL's mission to reduce poverty by ensuring that policy is informed by scientific

evidence. We assess mission-alignment using the following guiding questions. Not all of these questions

may be relevant to all proposals. Together, they reflect our broad conceptualization of what constitutes

a policy that can contribute to poverty reduction.

Study population

● Is the study population low-income or living in poverty?

● Is the study population one that has risk factors associated with falling into poverty?

CRRI RFP I

Proposals due: January 15, 2021

11

● Is the study population one that is at risk of living in or falling into poverty due to economic marginalization produced through structural racism in North America, such as Black, Indigenous, and People of Color (BIPOC)?

Policy Issue

● Does the policy issue have a significant or disproportionate impact on low-income and/or economically marginalized populations?

● Does the intervention improve or increase access to resources and choices for low-income and/or economically marginalized populations?

● Is the policy issue a broad, system-wide issue that affects many people, including low-income and/or economically marginalized populations?

● Is the policy issue an issue that, if unaddressed, could lead people to fall into poverty?

● Does the policy issue relate to issues of structural racism that economically marginalize populations in the North America context?

Examples: Connection to Mission Examples of Tested Interventions

Policy issue with a disproportionate impact on low-income populations

and/or economically marginalized populations

Nurse home-visiting program for first-time low-income mothers

Intervention that improves access to resources and choices for low-income

and/or economically marginalized populations

Access to a race concordant healthcare provider for Black men seeking preventive

health services

Broad, system-wide issue that affects many people, including low-income

and/or economically marginalized populations Medicare bundled payments for knee and hip replacements

Issue that, if unaddressed, could lead people to fall into poverty Program to buy and abolish medical debt; Blocking required disclosure of felony status

as a hiring input

Issue of structural racism that compounds or produces the economic

marginalization of specific populations, and increases their risk of remaining

or falling into poverty.

Informational cards for judicial decision-making that provide data on differential rates

of conviction for the same criminal charges across ethnic groups

A full list of funded evaluations by CRRI is available here. [LINK]

APPENDIX 2: ETHICAL RESEARCH AND RISK MITIGATION

J-PAL works to reduce poverty by ensuring that policy is informed by scientific evidence, and we hope

that the research conducted at J-PAL ultimately serves the communities who participate in it. Ethical

conduct of this research is therefore core to J-PAL’s mission, and it is of utmost importance that every

study is done ethically from start to finish.

J-PAL Research Protocols require that all projects obtain IRB approval and follow all procedures

approved by the IRB. However, approval by an IRB or other ethics commission does not necessarily

mean that all ethical dilemmas are resolved. IRB review cannot substitute for the researcher’s

responsibility to consider the ethical implications of their research and ask themselves as well as

members of the communities they work in if they are comfortable with their research protocols.

CRRI RFP I

Proposals due: January 15, 2021

12

The guiding questions below are meant not to duplicate the separate IRB application, but rather to fill

potential gaps. Many proposals may not need to respond to every question, but some proposals with

more sensitive interventions or contexts may need additional detail to ensure adequate responses.

Additional examples can be found at the bottom of this section.

1. What, if any, are the risks of unintended negative consequences of the research design, intervention, or potential misuse or misinterpretation of study results? Examples of risk points include:

a. The design of intake, consent, surveys, and randomization procedures b. Direct risks of the intervention. In the case of ‘nudge’ interventions, the program or

policy the study population is being ‘nudged’ to take up (Note: This question applies regardless of whether the research team is involved in designing or implementing the intervention).

c. Unintended consequences or spillovers to study participants, people outside the study, staff, and/or the research team caused by the intervention or study.

d. A policymaker intentionally or unintentionally interpreting null results as evidence of no impact, and using this to justify the termination of a potentially beneficial intervention. Null results on the harms of an intervention may justify the scaling of a potentially harmful intervention.

e. Evidence of heterogeneous impacts by race or gender being used to justify discrimination.

2. What steps has the team taken to assess, monitor, and mitigate/prevent any identified risks? Examples of risk assessment or mitigation strategies include:

a. Measuring welfare effects and potential impacts or spillover effects on the study population or those outside the study population, especially in the case that the risk of negative impact is (subjectively) high.

b. Planning to actively monitor for negative impacts. c. At the results sharing stage, planning to develop a thoughtful communications strategy

if the risk of harmful impacts of misinterpretation of the results seems high.

3. Why is this intervention appropriate in this context for this population? a. Does the population that will be intervened upon and/or whose outcomes will be

studied align with the population that stands to benefit from an application of the knowledge gained through this research?

b. Are there alternative interventions to consider that may have fewer risks and/or greater benefit? If so:

i. What about the alternatives, context, and/or counterfactuals make the proposed intervention appropriate?

ii. Would this intervention have happened without PI involvement? If not, how were the PIs involved in selecting or altering the chosen intervention?

CRRI RFP I

Proposals due: January 15, 2021

13

4. (For reviewers) Based on existing knowledge and research on related interventions or methods, is there reason to believe there may be risk of harm or other unintended negative consequences that the proposal did not identify?

a. Do you have concerns about the team’s awareness of this risk and ability to respond in a timely manner?

Examples:

These examples are meant to illustrate ethical concerns and mitigation plans that may go beyond what

an IRB would typically review, either because it involves risks to individuals who aren’t considered the

“subjects” of the research, or involves types of risks that may be difficult for an IRB to gauge without

specific subject-matter expertise.

Intervention Studied Concern & possible resolution

Free transportation during the COVID-19

pandemic

The research team could address concerns about the health risks of ridesharing during the COVID-19 pandemic

by sharing the rideshare company’s existing cleaning and mask wearing policies, as well as participants’ alternative

transportation options, in order to maximize their ability to make informed choices about risk.

Note: This scenario would likely already be covered by an IRB. Because we are actively tracking COVID-19 safety precautions

on projects, providing this risk mitigation plan may expedite further review of projects involving COVID-19 transmission risk.

A highly politicized program where results may

have strong impact on decision-making

The research team could plan to work closely with the implementing partner on all aspects of results

communication and dissemination in non-academic outlets, including proactively putting out policy briefs,

cooperating on press releases, and taking time to thoroughly explain results when contacted by policymakers.

Note: This is a good scenario, as we encourage policy-relevant research. All projects may have risk of misinterpretation. A

mitigation plan may be most needed when the risk level, or polarization around the topic, is particularly pronounced.

An opioid buyback program based at drop-off

centers in local pharmacies

The research team could address concerns about the risks to drop-off center staff by highlighting safety protocol

for staff

An intervention that requires adding an additional

form to apply to a public assistance program

The research team could recognize concerns of how this intervention could harm participants (e.g. by increasing

barriers to access caused by a higher administrative burden). The research team could provide detail on the policy

context surrounding the changing policy, their involvement (or lack thereof) in designing the policy change, how

this study could lead to improved outcomes for participants, and their plans to measure potentially adverse

effects.

Job support services that might have implications

for the general equilibrium of the broader labor

market

The research team could state their assumptions about why these services would / would not have adverse effects

on non-participants, and plan to measure general equilibrium / spillover effects.

A program that places law enforcement personnel

in educational settings to prevent gun violence

This program could be effective by pulling students engaging in risky behavior into the juvenile justice system,

and/or removing them from a school setting. Those pulled out may have been individuals who would have

committed acts of gun violence, but others may also be targeted and caught by this intervention. This would

create a risk of negative spillovers if the outcomes (e.g., welfare, education, future criminal acts) of the students

pulled out of school or into the justice system are worse than they otherwise would have been. This negative

impact may disproportionately affect students of color based on broader trends in educational discipline and in

policing. There may exist an alternative intervention that intended to prevent gun violence that does not have as

high of risks of negative impacts on some students (for example, cognitive behavioral therapy).

Researchers could address these concerns by: Describing evidence for why negative spillovers are minimal (i.e.,

that students pulled out of school or into justice system would have the same or better outcomes than without the

intervention); Measuring indicators of negative spillovers (e.g., arrest rates, expulsions, school completion) and

impact by racial or ethnic groups; Indicating if there is a threshold at which they would halt the study if the

measured harms are too high; Describing the policy and decision-making environment that made this particular

intervention preferable to study relative to other interventions to prevent gun violence.

Phased in “no exceptions” work requirements set

by a state legislature as a condition for accessing

public benefits

In this scenario, the new work requirements will be implemented regardless of researcher involvement. While

the evaluation will not change that, researcher involvement could potentially influence how the work

requirements are perceived by the general public even before demonstrating results, e.g. by “legitimizing” work

requirements as a policy that is likely to have more positive than negative trade-offs. Ultimately, depending on the

outcomes studied, the results may or may not fall in line with that assessment. Researchers could address this

possible discrepancy by describing why they feel that this study is in line with J-PAL’s mission even if the

intervention being examined results in decreased access to programs by people living in poverty, and by

CRRI RFP I

Proposals due: January 15, 2021

14

explaining how knowledge about this program is a net benefit even if their involvement impacts public perception

and support of the policy in a way that is counter to the best course of action suggested by their results.

APPENDIX 3: GRANT REQUIREMENTS

If funded, researchers agree to the following:

This grant is awarded at the discretion of the J-PAL the COVID-19 Recovery and Resilience Initiative

(CRRI) Review Board acting on behalf of the Massachusetts Institute of Technology (MIT). Funds are

to be used for the purpose described in the proposal narrative and proposal budget. If there are

significant changes to the scope, design, or budget of the project, grantees should contact the Initiative

Manager to obtain pre-approval for the proposed changes.

The terms of the award are as follows and as specified in any subaward established with MIT in

connection with the administration of the grant. Acceptance of funding from J-PAL North America

signals your consent to all these requirements. Non-compliance with these requirements could

affect your eligibility for future funding from any J-PAL initiative.

AEA and OSF Trial Registration and Pre-Analysis Plans

1. Researchers must register their trial with the AEA RCT Registry as soon as IRB approval or exemption is granted. For questions and support with the registry, please contact J-PAL staff member Keesler Welch (support@socialscienceregistry.org). Please send the assigned registration number to Keesler as soon as it is available, and copy Sarah Margolis (smargolis@povertyactionlab.org). Please note: Pilot projects that do not include any randomization (e.g. those that do NOT include an actual experiment) do NOT need to register.

a. Registration is required before the subaward can be established. b. Medical journals that are members of the International Committee of Medical Journal

Editors (ICMJE)--e.g., JAMA, NEJM--require registration on clinicaltrials.gov as a condition of publication. Registration on the AEA RCT Registry or OSF does NOT meet this requirement. We strongly urge researchers to register their trial on clinicaltrials.gov at least a week in advance of random assignment in order to meet this requirement, if there is any chance they will submit a paper to a medical journal.

2. Researchers must also create an account for their project on the Open Science Framework (OSF) as soon as IRB approval or exemption is granted. Researchers must 1) create an account, 2) create a public page for the project, and 3) add both “Keesler Welch” and J-PAL North America COVID-19 Recovery and Resilience Initiative (CRRI) as Administrators (do not check the “Bibliographic Contributor” box) and 4) link to your AEA RCT registration page. Please contact Keesler Welch (support@socialscienceregistry.org) with any questions.

a. Note: OSF registration and creation of a public page is required by J-PAL North America’s funder. J-PAL staff may add representatives from the funder as non-bibliographic contributors to the project page so that they can be added as an affiliated institution.

CRRI RFP I

Proposals due: January 15, 2021

15

b. Upon study completion, the AEA Registry and OSF pages must be updated with the results of the study and information on paper publication (as available).

3. All J-PAL North America funded studies must include a pre-analysis plan uploaded to either the AEA RCT Registry OR OSF. The pre-analysis plan must be uploaded prior to launch of the experiment (i.e. randomization activities). It may be made public or kept private if preferred. The pre-analysis plan must include the hypothesis or hypotheses to be examined in the empirical research study, the primary and secondary outcomes, and the statistical model(s) and methodologies to be used. Additional guidelines can be found here.

a. You will be asked to explain the status of your pre-analysis plan as part of the three-month start-up and annual report templates.

b. Failure to upload a pre-analysis plan prior to the experiment launching may jeopardize your eligibility for future funding.

Subaward Set-up

To set up your grant, you will need to submit the following document(s):

1. A formal letter of transmission from the subaward institution agreeing to the proposal and budget. The letter must be on the subaward institution’s letterhead, and should state that they support the proposal and plan to carry out the work in accordance with approved budget. They should also include contact information for the person who will receive and sign the subaward on behalf of their institution.

2. IRB Approval or Exemption: We expect that MIT’s IRB will not require a full review of your project but will instead cede authority to your university. If your institution uses the SMART IRB system (list of participating institutions), please submit your reliance request through this platform. Once the agreement is fully executed, please send the confirmation to Sarah Margolis. If your institution has not opted into this program, please send the following documents to Sarah Margolis (smargolis@povertyactionlab.org):

a. A copy of your university’s IRB approval or exemption and accompanying documents.

b. A 1-2 page summary of the research being conducted, written for a general audience;

c. Copies of any consent forms being used, with the hosting IRB’s approval stamp d. The name and title of the hosting IRB’s signatory official, FWA #, and IRB #

(the university’s IRB #, not the specific project’s IRB #) 3. Confirmation of AEA and OSF registration (as outlined above) should also be included in your

email to Sarah Margolis.

Once all materials have been received, it typically takes up to 60 days to establish the subaward,

though usually subawards can be backdated to allow costs from the date when all IRB approvals are in

place. The subaward will be paid on a cost-reimbursable basis.

Subaward spend-down

1. Invoices must be provided that show current expenses and cumulative spending by major line item (Salaries & Benefits, Travel, Field Costs, Equipment, Other Direct Costs), and cumulative totals

CRRI RFP I

Proposals due: January 15, 2021

16

should be compared to budget. The MIT subaward documents will provide an example, but please make sure your organization’s invoicing procedures follow this policy.

Please note that these invoices are subject to audit and MIT may request backup receipts. These cases are

rare but they do happen on occasion.

2. Invoices must be sent once per quarter at minimum or up to once per month. 3. Please note that J-PAL does not allow variance greater than 10% (by major line item) without prior

approval. We understand that budgets often change or need revisions during the lifecycle of a project and as such, we are happy to accommodate these modifications and they will typically be approved if the substance of the project is not changed. However, project staff are required to inform J-PAL and request approval in advance as we may need to modify the subaward agreement to be able to process invoices.

Reporting

1. Grantees must provide the reports below: a. a three-month start-up report to update J-PAL North America on your timeline and

provide information about pre-registration and pre-analysis plans (see template); b. a brief annual progress report highlighting key elements of RCT development such as

recruitment and enrollment due September 1 each year until the completion of the project (see template);

c. a final financial report within 60 days of completion of the award period (see template); d. a final project report with preliminary results within 4 months of completion of the

award period; i. We request that you submit a copy of any survey instruments used for the

project with your final report. Pilot studies are encouraged, but not required to share survey instruments.

e. The completed costing template (attached) with detailed information on program costs within 4 months of completion of the award period.

i. J-PAL has found that policymakers are very interested in how much a program costs, as it is one of the key factors in decision-making. Detailed cost data allows for cost effectiveness analysis (CEA), which J-PAL may conduct (with permission from the researchers).

ii. If researchers are unable to collect detailed cost data, we require initiative-funded projects to provide estimates of total program cost, average cost per beneficiary, and marginal cost to add another beneficiary.

iii. For questions about the scope of this requirement or assistance with the costing worksheet, or if you have a compelling reason for being unable to share any cost data, please email Caroline Garau (cgarau@povertyactionlab.org).

2. If you are not able to meet the reporting deadline you may request an extension. This request

should be submitted before the due date.  3. If projects are unable to complete the project by the end of the award period and need to extend

the award end date for the project, please fill out and submit a no cost extension request form. For questions about this please email Sarah Margolis at smargolis@povertyactionlab.org.

CRRI RFP I

Proposals due: January 15, 2021

17

4. Researchers whose reports are more than two months late and who have not received an approved

extension will not be eligible for additional J-PAL funding until the reports are submitted.  

Data & Code Publication

Except to the extent limited by law, IRB requirement, and/or any applicable binding agreement, we

require researchers to publish some or all of the data materials associated with the study (e.g. primary

data, program code, etc.) when one of the following conditions is met: within 60 days of an academic

papers’ acceptance (or by the deadline set by the journal in which the paper is accepted, whichever

comes earlier), or within three years of the completion of data collection. If some or all study data

cannot be shared for legal, ethical, or proprietary reasons, researchers must inform J-PAL in the final

narrative report. In such cases, J-PAL may waive the requirement to publish the data materials. Please

review J-PAL’s Data and Code Availability Policy for a detailed explanation of this requirement.

Paper Publication

1. After the completion of the RCT, the grantees will complete an academic paper and submit the academic paper to at least one peer-reviewed academic journal for publication. Grantees must send a PDF of the published paper to the Initiative Manager as soon as it is available, and report on intentions to publish in the final report.

a. If grantees are not able to do so, please inform the Initiative Manager and write a brief explanation to J-PAL detailing any challenges or issues that arose, which prevent grantees from writing a paper.

2. PIs are highly encouraged to share the public working papers from funded projects with Initiative staff as soon as they are available. J-PAL staff will then work with grantees to develop a strategy for communicating the results via traditional or social media and J-PAL publications like evaluation summaries.

J-PAL Involvement

1. All J-PAL affiliates and initiative invited researchers who are co-PIs on this project certify that they will be active, engaged and responsive PIs on this project dedicated to guaranteeing the quality control on all aspects of this research; and that their participation in this project is not merely to provide access to J-PAL resources and funding to anyone else working on this project who is neither a J-PAL affiliate, nor an initiative invited researcher.

2. J-PAL funded researchers are required to participate in one of J-PAL North America’s activities on a mutually agreed date and place. This activity could be a presentation of research, a matchmaking conference (bringing together researchers and implementing partners), a training (e.g. a lecture at J-PAL’s Executive Education course), or a presentation to one of J-PAL North America’s donors.

3. Applicants who are selected for funding will be requested to peer review proposals in future J-PAL North America RFP rounds.

4. The J-PAL academic network is asked to consider sharing data collection instruments and methodologies with other J-PAL affiliated researchers.

5. In an effort to gather and share lessons learned from funded RCTs, PIs are required to have a debrief call with J-PAL NA staff at the conclusion of the project. The purpose of this call is to learn from both the successes and challenges of implementing the project.

CRRI RFP I

Proposals due: January 15, 2021

18

Code of Conduct

Since J-PAL is part of MIT, everyone who is associated with J-PAL, including researchers worldwide

receiving grants from J-PAL initiatives, are considered part of the broader MIT community. Therefore,

it is our hope and expectation that they will adhere to MIT’s community-wide policies that are

available here. A part of MIT's broader policies, this section, titled "Relations and Responsibilities

Within the MIT Community," contains specific provisions regarding personal conduct, harassment,

discrimination and retaliation, violence against community members, and substance use. Please take

some time to review these.

Because almost all researchers we work with are also part of other university communities, they may

also be subject to their host universities’ policies and procedures. Many of these policies may be very

similar to the MIT policies above. Finally, many researchers are separately affiliated with other

academic associations and organizations, including the American Economic Association, and they

should continue to abide by the codes of conduct established by the associations and organizations to

which they belong. The AEA’s code of conduct is available here.

If anyone wishes to report that a researcher has violated MIT community policies, they should consult

the individuals and offices identified in the relevant policies linked above. In addition, all violations can

be directly reported to any of the following J-PAL contacts for further action: (i) Global Executive

Director; (ii) Mary Ann Bates (Executive Director J-PAL North America); (iii) David Sears(Global

Director of Finance and Operations); or (iv) Anna Omura (Global Senior Manager of Finance and

Operations).