teen pregnancy prevention program six-month progress ......the oah tpp program six-month progress...

OFFICE OF ADOLESCENT HEALTH (OAH) TEEN PREGNANCY PREVENTION (TPP)

PROGRAM

SIX-MONTH PROGRESS REPORT GUIDANCE

Last Updated November 2016

Six-Month Progress Report Due: February 1st

2

OAH TPP PROGRAM SIX-MONTH PROGRESS REPORT GUIDANCE

Table of Contents

PART ONE: GENERAL INSTRUCTIONS 3

PART TWO: PROGRESS REPORT SUBMISSION 3

PART THREE: PROGRESS REPORT CONTENT 3 I. Six-Month Progress Update 4 II. Performance Measures 6 III. Appendices 6 IV. Evaluation Reporting (for ALL TPP Grantees except Tier 2B) 6 V. Evaluation Reporting (Tier 2B Grantees Only) 6

PART FOUR: ADDITIONAL INFORMATION AND SAMPLE TEMPLATES 8

EXHIBIT A: Six-Month Progress Update Templates 9

EXHIBIT B: Description of OAH Grant Expectations 19

EXHIBIT C: MAX Filing Guidance for TPP Grantees – Year Two 24

EXHIBIT D: CONSORT Diagram Instructions, Templates, and an 27 Example (TPP Tier 2B Grantees Only)

3

OAH TPP PROGRAM SIX-MONTH PROGRESS REPORT GUIDANCE

PART ONE: GENERAL INSTRUCTIONS

The purpose of the six-month progress report is to report on the progress of the OAH TPP grant project during the six month project period (July 1 – December 31), including the status of programmatic activities, evaluation activities, and submission of performance measure data.

The OAH TPP Program Six-Month Progress Report Guidance describes the content and submission procedures for completing the six-month progress report. Six-month progress reports will be reviewed by the OAH Project Officer and the Office of Grants Management (OGM) Grants Management Specialist.

PART TWO: PROGRESS REPORT SUBMISSION

The six-month progress report and all supporting documents must be received no later than 11 p.m. Eastern Standard Time on February 1st.

Electronic Submission (required)

The six-month progress report, including all required documents, should be submitted to OAH and OGM electronically through Grantsolutions.gov. All required reporting documents should be submitted and uploaded within the Grant Notes section of “My Grants” for access by the assigned OAH Project Officer and OGM Grants Management Specialist. Grantees should include the grant number on all submissions. All materials must be submitted by February 1st at 11:00 p.m. Eastern Standard Time.

PART THREE: PROGRESS REPORT CONTENT

The six-month progress report should include: • Six-month progress update for the reporting period (July 1 – Dec 31) • Performance measure data for the reporting period • Additional materials in the appendices • Evaluation update for the reporting period

The contents of the report should be properly labeled and numbered. Content should be concise, complete and written in 12-point font.

4

I. SIX-MONTH PROGRESS UPDATE

The six-month progress update should include (see Exhibit A for the Six-Month Progress Update Template):

• An update on the status (i.e., met, unmet, in progress) and a narrative description of the progress in completing each of the OAH expectations for the respective grant program. (See Table 1 on page 5 for a summary of OAH grant expectations and Exhibit B on page 19 for a detailed description of the OAH grant expectations).

For each OAH expectation, you should describe the key activities that were implemented to address the expectation, as well as any accomplishments and barriers encountered. If the expectation has not been met, you should describe what is currently being done to meet the expectation and the timeline for when it will be met.

• An update on any other significant project activities that occurred during the six-month project period that don’t fall under one of the OAH expectations.

• A report on any significant barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.

• A description of key accomplishments during the six-month reporting period.

• Verification that all relevant documents from the reporting period (July 1 – Dec 31) have been saved in your Grantee Folder in Max.gov (see Exhibit C for the Max.gov Filing Guidance).

See Exhibit A for the Six-Month Progress Update Template for each TPP grant program. See Exhibit B for a Description of OAH Grant Expectations for each TPP grant program. See Exhibit C for the Max.gov Filing Guidance for TPP Grantees.

5

Table 1 – Overall OAH Grant Expectations for each TPP Grant Program

Capacity Building to Support Replication of EBPs (1a)

1. Project Management 2. Community Mobilization and Sustainability 3. Recruit and Retain CBA Recipients 4. Provide CBA to CBA Recipients 5. Safe and Supportive Environments 6. Youth-Friendly Healthcare Linkages and

Referrals 7. Dissemination and Communication 8. Evaluation 9. Continuous Quality Improvement

Replication of EBPs to Scale in Communities with Greatest Need (1b)

1. Project Management 2. Community Mobilization and Sustainability 3. Implementation of EBPs 4. Safe and Supportive Environments 5. Youth-Friendly Healthcare Linkages and


Supporting and Enabling Early Innovation (2a)

1. Project Management 2. Sustainability 3. Strategy to Foster Innovation 4. Innovator Selection 5. Innovator Progress and Materials 6. Innovator Supports 7. Dissemination and Communication 8. Evaluation 9. Continuous Quality Improvement

Rigorous Evaluation of New and Innovative Approaches (2b)

1. Project Management 2. Sustainability 3. Engage Target Population 4. Intervention Development and Packaging 5. Implementation 6. Safe and Supportive Environments 7. Youth-Friendly Healthcare Linkages and


Providing Capacity Building Assistance to OAH TPP Grantees (National CBA Providers)

1. Project Management 2. Coordination and Collaboration 3. Assess CBA Needs of TPP Grantees 4. Develop Coordinated CBA Plan 5. Provide CBA in Priority Area 6. Dissemination and Communication 7. Evaluation 8. Continuous Quality Improvement

6

II. TPP PERFORMANCE MEASURE REPORTING

All TPP grantees are required to submit their performance measure data for the period from July 1st through December 31st by February1st through the OAH TPP Performance Measure website (https://tpp.rti.org).

A summary of all required performance measures is available on the OAH TPP Performance Measure website (https://tpp. rti.org). All TPP grantees are expected to submit all required performance measure data for all youth served during the reporting period. In addition, all TPP grantees are expected to submit performance measure data on partners, training, and dissemination for the reporting period.

A Help Desk is also available if additional assistance is needed. To contact the Help Desk, click on the Help Desk tab at the top of the TPP Performance Measures Website (after logging on), and you will be able to contact our webmaster regarding your issue. When reporting your problem, please be as descriptive as possible by including the page on which the problem was encountered as well as steps that could be used to replicate the issue. In addition, please provide the name of your grantee organization along with your name, email, and telephone number.

III. APPENDICES

Supporting documents that add value or clarity to the information presented in the progress report should be included in the appendices. Materials included in the appendices should present information clearly and succinctly and add depth to your report.

IV. EVALUATION REPORTING (for ALL TPP Grantees except Tier 2B)

In addition to the update on evaluation and CQI activities provided in your progress report template, all TPP grantees should report any results to-date of their evaluation activities, including pilot results, lessons learned, and any results from implementation and/or outcome evaluation activities. Grantees should also report specifically on how they are using the data gathered through performance measures and evaluation activities to inform their decision-making and make continuous improvements to the grant project.

V. EVALUATION REPORTING (TPP Tier 2B Grantees Only)

All TPP Tier 2B grantees should submit an (1) evaluation abstract and (2) a CONSORT diagram as a part of their six-month progress report. The templates needed to complete these submissions are available at Max.gov on the Evaluation Information Page and as Exhibits at the end of this guidance.

The completed documents must be uploaded into Grantsolutions as a part of your six-month progress report. In addition, we ask that you also submit these evaluation reporting documents

https://tpp.rti.org/

https://tpp.rti.org/

https://community.max.gov/pages/viewpage.action?spaceKey=HHSExternal&title=TPP+Evaluation+Information

7

via email to [email protected] and copy your eval TA liaison and your OAH project officer.

Evaluation Abstract

The evaluation abstract should be a short (~2 page) structured summary of the intervention, evaluation design, sample, research questions and planned analyses. A template for this abstract is available here on the Max.gov website, along with an example of a completed evaluation abstract. See here for abstracts from the previous cohort. TPP Tier 2B grantees are asked to use the following convention to save the evaluation abstract: [Grantee Name]_Abstract_#.docx.

The TPP Eval team will provide feedback on your abstract before OAH formally approves it. Grantees should expect two rounds of review prior to approval. The Eval TA team will review submissions on a rolling basis to enable an efficient review process.

CONSORT diagrams

All TPP Tier 2b grantees need to provide information sample flow in a CONSORT diagram. A CONSORT diagram is a visual representation of the flow of sample members through an impact evaluation, including sample sizes from the time of random assignment through each of the data collection periods (e.g. baseline, first, and second [and if applicable, third] follow-up surveys).1 Frequently CONSORT diagrams are used in randomized controlled trials, but they are a valuable tool in understanding the sample flow and analytic sample in evaluations of any design.

If your study is using a cluster design, you will need to submit both an individual and a cluster CONSORT diagram. If your study is an individual design, you only need to complete an individual CONSORT diagram. Templates for the CONSORT diagrams are available here. The file should be named using the following convention: [GranteeName]_CONSORT_Feb 2017.docx.

Monitoring this aspect of your evaluation is important for understanding whether your implemented evaluation is maintaining the rigor of the original design. See the Attrition TA Brief for details on how non-response (attrition) can produce bias in an impact estimate. Understanding levels of and reasons for sample loss (attrition) provides information that can be used to target resources towards maximizing consent rates and response rates, either overall, by study condition, or for key subgroups. It is important to examine sample intake and follow-up data collection sample sizes throughout the study for two reasons: 1) to assess whether you are meeting the target sample size on which you based your power calculations, and 2) to assess the likelihood that the final analytic sample for key follow-up time periods will have rates of overall or differential attrition that exceed the HHS evidence standard threshold.2

1 It may also be useful to track non-consent that occurs after random assignment as a step in the CONSORT

diagram process for some designs.

mailto:[email protected]

https://community.max.gov/display/HHSExternal/Abstracts+-+Template+and+Examples

https://community.max.gov/download/attachments/1155794992/Cohort%202%20Abstract%20Example.docx?api=v2

http://www.hhs.gov/ash/oah/DEPRECATED/evaluation-original/granteeevals.html

https://community.max.gov/display/HHSExternal/Consort+Diagrams+-+Templates+and+Example

http://www.hhs.gov/ash/oah/oah-initiatives/assets/attrition-ta-brief.pdf

http://www.hhs.gov/ash/oah/oah-initiatives/assets/attrition-ta-brief.pdf

8

We recognize that many grantees are in the early stages of enrollment; therefore, sample sizes for the CONSORT diagram will likely be small and not representative of the final sample that will contribute to the evaluation report. However, completing the CONSORT diagram at this early stage of the evaluation will provide some preliminary information about sample flow, and will serve as useful practice in this activity that we will continue throughout the grant period.

EXHIBIT D includes a bulleted list of the information that the evaluator will need to complete the CONSORT diagram for either individual or cluster designs and an example of an individual level design and associated CONSORT diagram (this example is also available at Max.gov here).

The completed documents must be uploaded into Grantsolutions as a part of your six-month progress report. In addition, we ask that you also submit these evaluation reporting documents via email to [email protected] and copy your eval TA liaison and your OAH project officer.

PART FOUR: ADDITIONAL INFORMATION AND SAMPLE TEMPLATES

• EXHIBIT A: SIX-MONTH PROGRESS REPORT UPDATE TEMPLATES

• EXHIBIT B: DESCRIPTION OF OAH GRANT EXPECTATIONS

• EXHIBIT C: MAX FILING GUIDANCE FOR TPP GRANTEES – YEAR TWO

• EXHIBIT D: CONSORT DIAGRAM INSTRUCTIONS, TEMPLATEs, and EXAMPLE (Tier 2B Grantees Only)

https://community.max.gov/download/attachments/1155794980/Word%20template%20for%20CONSORT%20Diagrams_ind_revisedOct2016_example.docx?api=v2

mailto:[email protected]

9

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1A Grantees

Grantee Name Grant #:

July 1, 20xx – December 31, 20xx

OAH Grant Expectation STATUS In Progress, Met, Unmet

NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.

Project Management

Community Mobilization and Sustainability

Recruit and Retain CBA Recipients

Provide CBA to CBA Recipients

Safe and Supportive Environments

Youth-Friendly Healthcare Linkages and Referrals

Dissemination and Communication

Evaluation

Continuous Quality Improvement

10

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1A Grantees – p. 2

Update on Organizations Receiving CBA

For each organization receiving CBA, provide an update on the type of CBA received and progress in providing CBA

Organization #1 –

Organization #2 –

Organization #3 –

Additional Project Activities

Report on any other significant project activities that occurred during the first six-months of the project period.

Barriers, Challenges, and Solutions

Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.

Key Accomplishments and Lessons Learned for Six-Month Project Period

Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.

11

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1B Grantees





Project Management


Implementation of Evidence-Based Programs




Evaluation


12

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1B Grantees – p. 2







13

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2A Grantees





Project Management

Sustainability

Strategy to Foster Innovation

Innovator Selection

Innovator Supports

Innovator Progress and Materials


Evaluation


14

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2A Grantees – p. 2

Update on Selected Innovators

For each innovator receiving funding and support, provide an update on the innovator, the team, the interventions, and other relevant information.

Innovator #1 – update on innovators, the team, the intervention, other relevant information










Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome.



15

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2B Grantees





Project Management

Sustainability

Engage Target Population

Intervention Development and Packaging

Implementation




Evaluation


16

EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2B Grantees – p. 2







17

EXHIBIT A – Six-Month Progress Update Template for TPP National CBA Provider Grantees





Project Management

Coordination and Collaboration

Assess CBA Needs of TPP Grantees

Develop Coordinated CBA Plan

Provide CBA in Priority Area


Evaluation


18

EXHIBIT A – Six-Month Progress Update Template for TPP National CBA Provider Grantees – p. 2







19

EXHIBIT B: DESCRIPTION OF OAH GRANT EXPECTATIONS Included below is a list of expectations for each TPP grant program and a description of the specific grantee activities that fall within that grant expectation.

Capacity Building to Support Replication of EBPs (TPP Tier 1a)

OAH Grant Expectation

Description of Activities that Fall with the Expectation

Project Management

Recruit and retain staff; professional development for staff and partners; build, enhance, and sustain partnerships; monitor partners/ contractors to assess progress in meeting objectives; assess community needs


Assist CBA recipients in community mobilization; sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)

Recruit and Retain CBA Recipients

Recruit (if applicable) and retain youth-serving organizations to receive capacity building assistance

Provide CBA to CBA Recipients

Assess needs of youth-serving organizations; develop a plan for providing CBA to meet organization needs; provide CBA; revise, update, and implement activities with CBA recipients; monitor CBA recipients to assess progress in meeting objectives and to ensure implementation of EBPs


Annually complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; identify strategies for ensuring programs are implemented in safe and supportive environments for youth and their families


Identify and recruit youth-friendly healthcare providers; assess providers to ensure services are youth-friendly; develop protocols and procedures for making referrals; develop and disseminate provider referral guide; identify and train staff on referral protocols and procedures


Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences

Evaluation Collect, report, and use performance measure data; evaluate the implementation and success of CBA provided to youth-serving organizations; evaluate the success of each CBA recipient’s efforts to implement EBPs


Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project

http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/training/Assests/traumainformed-checklist.pdf


http://www.hhs.gov/ash/oah/oah-initiatives/assets/tpp-grantee-orientation/guide-for-lgbtq-inclusivity.pdf


http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/training/Assests/pyd-tpp-checklist.pdf


20

Replication of EBPs to Scale in Communities with Greatest Need (TPP Tier 1b)


Description of What is Included Within Expectation

Project Management

Recruit and retain staff; professional development for staff and partners; monitor partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships; continue to assess community needs; use Getting to Outcomes® to plan, implement, and evaluate the project


Mobilize the community to develop, implement, and continuously improve the plan to prevent teen pregnancy and promote positive youth development in their community; develop and continue to engage the Community Advisory Board in each community; develop and continue to engage the Youth Leadership Council in each community; sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)

Implementation of EBPs

Implement EBPs to scale in all settings (at least 3) in each community served; recruit, retain, and engage participants; monitor implementation of programs with fidelity and quality


Annually complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; identify strategies for ensuring programs are implemented in safe and supportive environments for youth and their families (i.e., at least 3 actions for TIA, at least 1 action for each of the six constructs for inclusivity, and at least 3 actions for PYD)




Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences

Evaluation Collect, report, and use performance measure data; conduct implementation evaluation to document the process for developing and implementing the program and to identify key successes, challenges, and lessons learned; conduct outcome evaluation to compare outcomes in the community(ies) served to outcomes in a similar community that did not receive the program









21

Supporting and Enabling Early Innovation (TPP Tier 2a)

OAH Grant Expectation Description of What is Included Within Expectation

Project Management Recruit and retain staff; professional development for staff and partners; monitor implementation partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships

Sustainability Sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)

Strategy to Foster Innovation

Implement a promising strategy to foster innovation using a framework, theory of change, or other strategic approach

Innovator Selection Convene an open, national competition that results in selecting, funding, and retaining innovators

Innovator Supports Provide support to innovators, both individual and group-based

Innovator Progress and Materials

Monitor innovator progress; ensure that innovator materials are medically accurate, age-appropriate, culturally appropriate, LGBTQ inclusive, and trauma-informed


Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences.

Evaluation Collect, report, and use performance measure data; monitor fidelity for the project; evaluate the implementation and success of the project; evaluate the success of each innovator



22

Rigorous Evaluation of New and Innovative Approaches (TPP Tier 2b)

OAH Grant Expectation Description of What is Included Within Expectation

Project Management Recruit and retain staff; professional development for staff and partners; monitor implementation partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships

Sustainability Sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)

Engage Target Population Involve target population in the development, implementation, and evaluation of the grant project

Intervention Development and Packaging

Develop, pilot, and refine the intervention; package the intervention to be implementation ready by the end of the grant using the OAH TPP Packaging and Dissemination Guidance

Implementation Implement the intervention; recruit, retain, and engage participants; monitor implementation of intervention and control with fidelity and quality


Complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; use the results to identify strategies for ensuring programs are implemented in a safe and supportive environment for youth and their families




Develop, maintain, and implement a dissemination and communications plan; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences.

Evaluation Collect, report, and use performance measure data; conduct rigorous evaluation of program implementation and program outcomes



http://www.hhs.gov/ash/oah/oah-initiatives/tpp_program/program-guidance/Assets/tpp_packaging_guidance.pdf







23

Providing Capacity Building Assistance to OAH TPP Grantees (National CBA Providers)


Description of What is Included Within Expectation

Project Management

Recruit and retain staff; training and professional development for staff and partners; monitor partners and/or contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships

Coordination and Collaboration

Coordinate and collaborate with the other national CBA providers and with OAH to regularly assess grantee needs, develop an annual coordinated CBA plan, provide effective CBA designed to meet grantee needs, and eliminate duplication and streamline services for individual TPP grantees

Assess CBA Needs of TPP Grantees

Assess the TPP grantee needs for CBA in the five key priority areas

Develop Coordinated CBA Plan

Use the results from the assessment of TPP grantee needs to develop an annual coordinated CBA plan that will guide the development and provision of CBA for the year

Provide CBA in Priority Area

Provide CBA that is tailored to grantee needs and uses diverse approaches; provide three levels of CBA to TPP grantees (low, medium, and high intensity), ensuring that all TPP grantees receive some level of CBA from each CBA provider annually; ensure CBA materials are medically accurate, age appropriate, culturally and linguistically appropriate, and inclusive of LGBTQ youth


Develop, maintain, and implement a dissemination and communications plan; disseminate and communicate information about project activities, lessons learned, successes, and evaluation results

Evaluation Collect, report, and use performance measure data; evaluate the implementation and success of the CBA provided to TPP grantees



24

EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO

The OAH Project Management and Collaboration Community (Max.gov) is used to: 1. Track and submit requests related to your grant project in MAX Forms; and 2. Store important documents for your grant project in MAX Folders to create a shared,

historical record of the project.

For MAX Forms, each grantee has a secure space to submit requests and track information. MAX Forms is where you are expected to submit:

• grantee profile updates, • medical accuracy requests (note - medical accuracy review documentation must also be

submitted in the MAX folders); • major adaptation requests, and • minor adaptations notices.

For MAX Folders, each grantee has a separate grantee folder, also called the grantee’s individual page. The grantee and OAH Project Officer should work collaboratively to ensure the grantee folder is up-to-date and includes all relevant files, in accordance with the filing guidance below. Documents should be uploaded as an attachment to specific folders as indicated below. Grantees must consult with their Project Officers before creating any additional subfolders within their MAX folder.

MAX Folders for Tier 1A Grantees

MAX Folder Pages What to Upload Dissemination and Communications

All press, photos, communication plans, etc.

Evaluation and CQI All documents related to evaluation and continuous quality improvement. Medical Accuracy Review

All documents for the internal medical accuracy review, OAH’s medical accuracy review, program materials, and insert pages/modifications. Please note grantees must also submit the MAX Form when submitting a medical accuracy review request.

MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1 – e.g. Materials Review, CBA Plan, etc.

Documents should be uploaded to sub-folders as appropriate; otherwise, attach them to the “Year 1” page.

Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Community Mobilization and Sustainability 3. Recruiting and Retaining CBA Recipients 4. Providing CBA to CBA Recipients 5. Safe and Supportive Environments 6. Youth-Friendly Healthcare Linkages and Referrals

https://community.max.gov/display/HHSExternal/OAH+Project+Management+and+Collaboration+Community

https://community.max.gov/display/HHSExternal/TPP+Forms

https://community.max.gov/display/HHSExternal/Teen+Pregnancy+Prevention+%28TPP%29+Grants

25

EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO (cont. p. 2)

MAX Folders for Tier 1B Grantees



Evaluation and CQI All documents related to evaluation and continuous quality improvement. Medical Accuracy Review


MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1 – e.g. Materials Review, Pilot Results, etc.

Documents should be uploaded to any sub-folders as appropriate; otherwise, attach them to the “Year 1” page.

Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Community Mobilization and Sustainability 3. Implementation of EBPs 4. Safe and Supportive Environments 5. Youth-Friendly Healthcare Linkages and Referrals

MAX Folders for Tier 2A Grantees



Evaluation and CQI All documents related to evaluation and continuous quality improvement Medical Accuracy Review


MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1. Documents should be uploaded to any sub-

folders as appropriate; otherwise, attach them to the “Year 1” page. Year 2 All other documents from Year 2 including those related to:

• Selection • Strategy • Support • Innovations

26

EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO (cont. p. 3)

MAX Folders for Tier 2B Grantees



Evaluation Technical Assistance and CQI

All documentation related to: • Abstract • Impact analysis plan • Implementation analysis plan • Monthly call notes • IRB • Continuous quality improvement

Upload documents to the sub-folders as appropriate; otherwise, attach them to the “Evaluation Technical Assistance & CQI” page.

Medical Accuracy Review


MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents related to Year 1 – e.g. Materials Review, Pilot Results,

etc. Documents should be uploaded to any sub-folders as appropriate; otherwise, attach them to the “Year 1” page.

Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Sustainability 3. Engage Target Population 4. Intervention Development and Packaging 5. Implementation 6. Safe and Supportive Environments 7. Youth-Friendly Healthcare Linkages and Referrals

27

EXHIBIT D CONSORT Diagram Instructions, Templates, and an Example (Tier 2B Grantees Only)

Instructions

CONSORT DIAGRAM documentation

All CONSORT diagrams requires two pieces of information as context for interpretation:

1. A “time stamp” for the CONSORT diagram. The CONSORT should provide information about the number of clusters (or individuals) randomized and the number of clusters (or individuals) participating (i.e., that have not dropped out) as of this particular point in time. Because clusters or individuals may be at different stages of the study at any point in time due to rolling enrollment or multiple cohorts of implementation, certain clusters or individuals may not have an opportunity to contribute data to all data collection points (see more on this below).

2. A brief paragraph (~3-4 sentences) describing the design and data collection, including full enrollment period (start/end dates) and program start/end dates.

CONSORT diagrams that track clusters as the unit of interest For clustered random assignment designs (e.g., random assignment of clinics, community-based organizations, teachers, or schools) and quasi-experimental designs.

The following pieces of information are also necessary as documentation of sample flow in cluster-level CONSORT diagrams:

• The number of clusters (randomly) assigned to each condition (i.e., treatment and comparison).

• The number of clusters still participating (i.e. retained), by study condition, at each data collection time point. A cluster is defined as “participating” if at least one individual in the cluster completed the data collection event.

o At any given time point, some clusters may not have had the opportunity to contribute data for a particular data collection event. For example, clusters with youth that have not completed the program would not have had the opportunity to contribute follow-up data. Therefore, it is important to document the number of clusters that are eligible (i.e. the number of clusters that could have had youth contribute data) at a given time point, in addition to providing the number of clusters that actually did have youth provide data (i.e. the number of cluster retained).

• In addition to tracking cluster flow, it is necessary to complete a CONSORT diagram for the youth in participating clusters. The HHS evidence standards focus on attrition among those clusters who participate in each data collection event; therefore, the randomized sample in the individual- level CONSORT diagram should only include youth in the clusters that are still participating in the evaluation. More on this below.

28

EXHIBIT D (cont. p. 2)

CONSORT diagrams that track individuals as the unit of interest For both individual level and cluster level designs.

The following pieces of information are also necessary as documentation of sample flow in individual- level CONSORT diagrams:

• The number of youth (randomly) assigned in total, the number assigned to each condition, and the start and end dates of assignment.

o If the evaluation uses a cluster design, then the number of youth in each condition at any time point should reflect the number of youth in participating clusters at that time point.

• The number of youth that have provided data, by study condition, at each data collection time point (baseline and subsequent follow-ups).

o At a given time point, a subset of individuals may not have had the opportunity to contribute data for a particular data collection event. For example, individuals that have not completed the program would not have had the opportunity to contribute follow-up data. Therefore, it will be important to document the number of youth that are eligible (i.e. the number of youth that could have contributed data) at a given time point, in addition to providing the number of youth that actually did provide data.

Templates Templates for the CONSORT diagrams are available on the Max.gov website here.

CONSORT DIAGRAM example

Individual level design This example describes sample flow in an individual RCT, where youth are randomly assigned to treatment and control conditions on an on-going basis. The study began on January 2, 2016, with random assignment continuing through the end of December 2017. The data collection periods for this study are baseline, immediate post-intervention, and 6-months post-intervention. The CONSORT diagram represents sample sizes through July 15, 2016.

As of July 15, 2016, 1,000 youth had been randomly assigned to condition at some point since the study began on January 2, 2016. In total, 500 youth were assigned to the treatment condition and 500 were assigned to the control condition. All randomly assigned youth completed the baseline survey. Only a subset of the youth had been in the study long enough to be eligible to complete this first follow-up. Specifically, 450 youth in each condition. The other 50 youth in each condition, who were recently randomly assigned to condition, had not completed the program (and thus, were not eligible to complete the first follow-up survey). Among the 900 youth (450 in each condition) who had the opportunity to complete the first follow-up, 445 members of the treatment group and 442 members of the control group completed the survey. Finally, a very small subset of the sample (only 50 youth in each condition) had been in the study long enough to have the opportunity to complete the 6-month follow-up survey. Of these 100 youth, 45 members of the treatment group and 48 members of the control group completed the survey. The majority of the study participants (450 youth in each condition) had not yet been in the study long enough to be eligible for the 6-month follow-up survey.

https://community.max.gov/display/HHSExternal/Consort+Diagrams+-+Templates+and+Example

29

EXHIBIT D (cont. p. 3)

EXAMPLE - CONSORT Diagram for Individual-Level Sample Flow

Data included in report are current as of:

July 15, 2016

This evaluation is an individual-level RCT with youth assigned to treatment or control. They study will collect data baseline, immediate post-intervention, and 6-month follow-up data. Youth are enrolled and randomly assigned on a rolling basis between January 2016 and December 2017. Programming will take place over the same period, with all programming ending in January 2018.