Your Role in Ensuring Equitable Reviews

2014 AmeriCorps External Review Training

2

Training ObjectivesBy the end of this training, you should be able to:¨ Describe the Review Participants’ responsibilities

in ensuring a fair and equitable review¨ Outline the steps to take if you identify a

potential Conflict of Interest¨ Explain examples of potential bias¨ List proper disposal methods for all confidential

materials¨ State the purpose of the “Confidentiality and

Conflict of Interest Statement” for Review Participants

¨ Discuss the recognition and value of diversity in programs

¨ Discuss the importance of page limits for applications

Preview of Assessment Questions(Multiple Choice)¨ You find yourself reviewing an application in

which your organization will be a sub-recipient of an applicant organization. This is an example of:

(Short Answer)¨ Briefly state your responsibility in ensuring equity

in reviews in the following areas: Page Limits, Program Models, and Bias.

3

4

Equity in the External Review¨ Ensuring that each application receives

consideration through a fair and equitable process that preserves the transparency and integrity of the grant application review

¨ Understanding the Selection Criteria and Standards Assessing the sections and applying ratings

appropriately Balancing the value of your expertise, professional

opinion Appreciating the variety and diversity of programs Utilizing the Review Rubric Using only the materials given to you; not considering

outside information or factors (reputation, knowledge of organization, hyperlinks)

¨ Shared responsibility as a Review Participant Conflicts of Interest, Bias, & Confidentiality Page limits

5

Conflict of Interest¨ Conflict between private interests and official

responsibilities¨ CNCS considers both direct and indirect COIs¨ Consider those around you in your daily lives

6

Examples of Conflicts of Interest:Direct and Indirect

¨ You are assigned to review an application for an organization for which: Your business colleague is serving on the Board of

Directors Your sister has been volunteering for 10 years You provided consulting services in 2011 Your daughter has applied for employment You are currently employed by an applicant/affiliate

organization Your friend’s business could benefit financially Your organization will be a sub-recipient of an applicant

organization You no longer work for the applicant org as of 2005, but

were employed there for 15 years

7

Importance of Reporting COIs¨ Ensure a fair and equitable review for each

application¨ Preserving the integrity of the External Review

process

¨ If you have any questions or think a conflict may exist, immediately contact your Panel Coordinator and GARP Liaison

¨ CNCS will determine the appropriate steps, if any, regarding your COI concern

8

Understanding Potential Bias¨ What is bias?

A preference or inclination that may inhibit impartial judgment Can be for or against

¨ Examples of potential bias can include: Agreements/disagreements with methods or models in the

program without basis Favor or dislike of the author or applicant Consideration of outside information (positive or negative) that

is not included in the application¨ How should instances of bias be handled?

Often will be flagged by another participant (based on keen observations)

Return to assessment and provide facts only to reconsider assessment

Exercise consideration and respect, remove emotions Possible recusal from review of that application

9

Confidential Information

Applicant Confidentiality

• Applicant names• Applicant business

information and financial data

• Details about the proposed program

• Review comments and review ratings

Participant Confidentiality

• Identity of Review Participants during the review

• The link between Reviewers and their comments

10

Proper Handling of Confidential Items

Reviewer Names

All Review Notes

All Applications

Review Artifacts

11

Confidentiality & COI StatementProcess and Purpose

Read the Confidentiality and COI Form

Sign and return to CNCS

Receive the applications

Flag any potential COIs (after start of review)

Maintain appropriate confidentiality

To ensure you are aware of your responsibility

To verify shared understanding and expectations for CNCS Standards in External Reviews

12

Diversity in Programs

Focus Areas Types of Organizations

Scope of Proposed Project/

Program Size

Program Model Target Population

Page Limits

13

15 Page Application Narrative Limit

• Budget• Performance Measures• Evaluation Plan Narrative, or• Required Supplementary

Materials

The narrative page limit DOES NOT include:

The narrative page limit includes:• SF 424 Facesheet• Executive Summary• Program Design • Organizational Capability • Cost Effectiveness and Budget

Adequacy

14

Wrap Up¨ Describe Review Participant and CNCS

responsibility in Conflict of Interest (COI)¨ Restate steps to take if you identify a potential

COI¨ Understand examples of potential Bias¨ List proper disposal methods for all confidential

materials¨ Understand the purpose of the Confidentiality

and Conflict of Interest Statement for Review Participants

¨ Recognize and value the importance of Diversity in Programs and Page Limits

Understanding the CNCS Grant Application Review

Process

2014 AmeriCorps External Review Training

16

Training Objectives

¨ Identify the steps in the life cycle of CNCS competitive grants

¨ Describe the steps of the CNCS External Review process

¨ Define the roles and responsibilities of Review Participants

¨ Describe eGrants and how it is used

17

Overview of the CNCS Grant-Making Process

18

Corporation for National and Community Service

¨ Created in 1993, and expanded opportunities for Americans to serve their communities through national service

¨ Nation’s largest grant maker supporting service and volunteering

¨ Engages millions of Americans of all ages and backgrounds in service each year through its programs

¨ Promotes United We Serve: www.serve.gov ¨ For more information, please review the CNCS

Strategic plan: http://go.usa.gov/Z6eF

19

Signed on April 21, 2009, the Serve America Act went into effect on October 1, 2009

More information on the Serve America Act can be found at:http://go.usa.gov/Z6tT Edward M. Kennedy Serve America Act

2009

Corporation for National and Community Service

20

Close Out

Competitive Grant Life Cycle

Monitor &Administer

Grant Award Grants Notification of

AwardsCNCS Award

Decision

Receive Applications

Solicit Applicants &External Reviewers

Plan for Grant Competitions

Congress Appropriates Funding

REVIEW APPLICATIONS

21

Overview of the Grant Application Review Process

(GARP)

Send final IRFs (Word doc) to GARP Liaision

Enter IRFs and AFSs in eGrants

Complete evaluation

External Review Process

22

Read the application Provide a rating for each

element (R) Comment on significant

strengths and weaknesses (R) PC reviews & provides

Feedback

Edit Individual Reviewer Forms (IRFs) (R)

Compile Applicant Feedback Summaries (AFS) (PC)

Receive Feedback on AFS from Program Officer Liaison

Submit FinalIndividual Reviewer

Forms (IRFs)

Participate inPanel Discussions Assess Applications

Review Applications for COI

Download Assigned Applications

Review Training Materials

Complete Close Out Process

23

Roles and Responsibilities¨ Review Participants:

External Reviewer Conducts assessments of and provides ratings for each application Participates in Panel Discussions Completes high-quality Individual Reviewer Forms (IRFs) Incorporates PC Feedback to improve IRF Three Reviewers on the Panel. At least one Reviewer will have

Evaluation experience. Panel Coordinator

Manages panel logistics and facilitates the Panel Discussions Provides feedback to each Reviewer on every IRF Completes the Applicant Feedback Summary for each application

discussed Compiles the Justification of Evidence Level and Quality form Completes Panel Coordinator Notes for each application

Roles and Responsibilities (cont.)¨ Grant Application Review Process (GARP)

Liaison Primary contact for process-related guidance and logistical

support Checks with each Reviewer and PC on review progress to assist

¨ Program Officer Liaison (POL) Provides CNCS programmatic expertise and guidance Available as a resource for clarification of Criteria and

Standards Reviews the Applicant Feedback Summaries to ensure

comments relate to Selection Criteria

24

Key Review Forms¨ Individual Reviewer Form (IRF)

Captures the External Reviewers’ assessments and the significant strengths and weaknesses identified in an application

Completed by External Reviewers for each application

¨ Applicant Feedback Summary Compilation of the External Reviewers’ comments

for Applicant Feedback (Strengths and Weaknesses from each IRF)

Compiled by the Panel Coordinator¨ Justification of Evidence Level and

Quality Prepared by Panel Coordinator after the IRFs are

final Informs Internal Review (not for Applicant

Feedback)

25

eGrants You will access eGrants at three junctures in the Review—¨ Download applications

¨ Enter banking information

¨ Enter review results (in addition to the final Word product)

26

Expectations of Review Participants

¨ Review all orientation and training materials 2014 AmeriCorps Peer Review Handbook Orientation Sessions Notice of Federal Funding Opportunity (NOFO)

¨ Report any actual or potential Conflicts of Interest (COI) and comply with confidentiality expectations

¨ Produce high-quality review products¨ Be responsive to requests from the GARP Liaison

and POL¨ Satisfactorily complete all Review Participant

responsibilities27

Wrap Up and ReviewYou should now be able to:¨ Identify the steps in the life cycle of CNCS

competitive grants¨ Describe the steps of the CNCS External Review

process¨ Define the roles and responsibilities of Review

Participants¨ Describe eGrants and how it is used

28

Assessment¨ To check for understanding  and verify that you

have completed this orientation session, please complete the Assessment at the following link: https://www.surveymonkey.com/s/equitablereview

29