the state of high school equivalent program (hep) evaluation u.s. department of education office of...

The State of High School Equivalent Program (HEP) Evaluation

U.S. Department of Education Office of Migrant Education Webinar

August 22, 20132:00 pm – 3:30 pm EDT

Overview

I. Background and PurposeII. OME Review of Evaluation ReportsIII. Rubric for ReviewIV. Evaluation Reports ReviewedV. ObservationsVI. Notable Evaluation ReportsVII. Initial Grantee FeedbackVIII. Today’s Feedback IX. Next Steps

2

I. Background and Purpose

3

I. Background and Purpose

• Compliance• Improvement

4

Compliance: EDGAR

• EDGAR -- Education Department General Administrative Regulations

• 34 CFR §75.590 requires a grantee to evaluate annually: – The recipient’s progress in achieving the objectives in its

approved application;– The effectiveness of the project in meeting the purposes

of the program; and – The effect of the project on served participants

5

Compliance: HEP Application Evaluation Plans• Each approved and funded grant application included an

evaluation plan within the Narrative and Quality of Project Evaluation (Section 7)

• “…In determining the quality of the evaluation the Secretary considers the following factors: – (i.) The extent to which the methods of evaluation are appropriate

to the context within which the project operates. (3 points possible)

– (ii.) The extent to which the methods of evaluation provide for examining the effectiveness of project implementation and strategies. (3 points possible)

– (iii.) The extent to which the methods of evaluation will provide performance feedback and permit periodic assessment of progress toward achieving intended outcomes. (4 points possible).

6

Compliance: HEP Application Section 7: Note

Quality of the Project Evaluation:• Important note about the project evaluation: A strong

evaluation plan should be included in the application narrative and should be used, as appropriate, to shape the development of the project from the beginning of the grant period. The plan should include benchmarks to monitor progress toward specific project objectives and also outcome measures to assess the impact on teaching and learning or other important outcomes for project participants. More specifically, the plan should identify the individual and/or organization that have agreed to serve as evaluator for the project and describe the qualifications of that evaluator. (continued)

7

Compliance: Application Section 7: Note, continued

The plan should describe the evaluation design indicating:• (1) What types of data will be collected• (2) When various types of data will be collected• (3) What methods will be used• (4) What instruments will be developed and when • (5) How the data will be analyzed• (6) When reports of results and outcomes will be available• (7) How the applicant will use the information collected through the

evaluation to monitor progress of the funded project and to provide accountability information both about success at the initial site and effective strategies for replication in other settings.

• Applicants are encouraged to devote an appropriate level of resources to project evaluation

8

Compliance: APR: Evaluation References

• Annual Performance Report (APR) – requirement for grantees• Includes evaluation material:

– APR Section D Project Goals and Objectives– Section 2) Explanation of Progress (Includes Qualitative Data and

Data Collection Information) maximum 2500 words1. For each project objective and associated performance measures, indicate

what data (quantitative and/or qualitative) were collected and when they were collected, the evaluation methods that were used, and how the data were analyzed. Clearly identify and explain any deviations from your approved evaluation plan, including changes in design or methodology, or the individual or organization conducting the evaluation….

4. Indicate how you used your data and information from your evaluation to monitor the progress of your grant, and if needed, to make improvements to your original project plan (e.g., project activities and milestones) which are consistent with your approved objectives and scope of work.

9

Improvement: Continuous Improvement Cycle

Research and Development of HEP Application

(Plan)

Guide

Implementation of HEP

Application Services and

Activities(Do)

Formative/ Summative

Program Evaluation

(Study)

Modify Services and Activities to

Improve Performance/

Develop new HEP Application

(Act)

Incor-

porate

10

II. OME Review of Evaluation Reports

11

II. OME Review of Evaluation Reports

OME initiated a review of evaluation reports received from HEP and CAMP grantees:

• Requested current grantees submit evaluation reports by April 30, 2013

• 25 HEP and CAMP evaluation reports were submitted to OME

• Evaluation reports addressed periods of review ranging from 2009 through 2012

12

HEP Grantee Institutions and Agencies

CURRENT GRANTEE INSTITUTIONS AND AGENCIES

Type of Grantee Institution of Higher Education Non-IHE Agencies Combination Total

HEP 38 9 0 47

13

III. Rubric for Review

14

III. Rubric for Review

Rubric for review of evaluation reports:• Developed to permit consistent review of all submitted

evaluation documents • Identified core information of particular interest

to OME, based on approved applications• Scored on a general scale:– “0” – Does Not Meet Expectations– “1” – Needs Improvement– “2” – Meets Expectations– “3” – Exceeds Expectations

15

Rubric for Report Review, continued

• Scoring of components 1 – 5 was influenced by degree evaluation documents provided qualitative and/or quantitative data

• Maximum score possible: 108 • Average HEP evaluation report score: 40.6

16

Rubric: Outline of Components

Rubric incorporated 5 components:• Component 1 – GPRA Results• Component 2 – Fidelity of Implementation to Design• Component 3 – Effectiveness of Project Design• Component 4 – Collaborative Agreements• Component 5 – Recommendations

17

Rubric: Component 1 – GPRA 1 and GPRA 2

• Evaluation report scored in Component 1 on:– 1.a.b. Presentation of GPRA 1 and GPRA 2 results and

related progress to meeting performance requirements– 1.c. Presentation of performance measures included in

grantee’s approved application and target v. actual results– 1.d. Information on data validation for GPRA 1 and GPRA

2 results • Maximum points available – 12 points

18

Rubric: Component 2 – Fidelity of Implementation to Design

• Rubric Component 2 scored evaluation report’s qualitative or quantitative data for grantee level of fidelity to implementation of: – 2.a. Instructional Services Design– 2.b. Support Services Design– 2.c. Placement Services Design– 2.d. Management Plan Design– 2.e. Recruitment Plan Design

• Maximum points available – 15 points

19

Rubric: Component 3 – Effectiveness of Project Design

• Component 3 details areas in Component 2• Evaluation reports scored for information on specific

topics including:– Review of eligibility screening tools– Use of student/staff surveys– Staff qualifications– Student stipends– Planned against actual costs– Student demographic information


20

Rubric: Component 3.a. – Effectiveness of Project Design, continued

• 3. Effectiveness of Project Design– 3.a. Instructional Services Design• 3.a.1. The report includes data on the number of

instruction hours provided and schedule/availability of instruction hours to meet individual needs.• 3.a.2. The report includes data on the type and usage of

eligibility screening tools.• 3.a.3. The report includes data on how instructional

requirements of individual students are assessed and met.

21

Rubric: Component 3.b.-3.d. – Support, Placement, Management

– 3.b. Support Service Design– 3.c. Placement Services Design for HEP projects– 3.d. Management Plan Design. Analyzes the

effectiveness of the grantee’s management plan design• 3.d.1. Teaching and administrative staff structure• 3.d.2. Senior Project administrative staff• 3.d.3. Qualifications of project instructors• 3.d.4 Professional Staff development• 3.d.5 Student Records• 3.d.6 Approved Procedures Policy Manual

22

Rubric: Component 3.d.-3.e. – Management, Recruitment

• 3.d.7. Financial/cost sharing issues with host institutions• 3.d.8. Student stipends• 3.d.9. Planned and Actual costs

– 3.e. Recruitment Plan Design Report analyzes the effectiveness of the grantee’s recruitment plan design• 3.e.1. Demographic information• 3.e.2. Training for recruiters

23

Rubric: Component 4 – Collaborative Agreements

• In Component 4, evaluation reports scored based on information and analysis that addressed:– 4.a. Existing collaborative agreements with educational

institutions or service providers– 4.b. Planned collaborative agreements– 4.c. Relationships with host institutions, including

unresolved facilities, equipment/computers, HR and related issues that affect project performance


24

Rubric: Component 5 – Recommendations

• In Component 5, evaluations scored on recommendations for key areas that were linked to information included in the report: – 5.a. Instructional Services– 5.b. Support Services– 5.c. Placement Services – 5.d. Management Plan– 5.e. Recruitment Plan


25

IV. Evaluation Reports Reviewed

26

HEP Evaluation Reports

• 15 evaluation reports submitted from 14 grantees• Reports submitted from 10 states:

Arizona, Arkansas, California, Kansas, Maine, New Jersey, Oklahoma, Pennsylvania, Texas, and Washington

TYPE OF GRANTEE INSTITUTIONS

Type of Grantee Institution of Higher Education Non-IHE Agencies Combination

HEP 10* 4 0

* One grantee submitted documents for two different periods of performance that were scored as separate evaluations

27

Type of HEP Evaluations Submitted

• Wide range of evaluation reports• Several reports focused specifically on:

– Questionnaires– Student surveys– Site visit reports– Focus group results

• Some reports were detailed, 15-30 page formative or summative evaluations with numerous appendices

• Each document submitted to OME was reviewed through the rubric

28

Type of HEP Evaluations Submitted

TYPE OF HEP EVALUATION DOCUMENTS SUBMITTED(as defined by evaluator and/or noted in document)

Type of Grantee

Number of Distinct Grantees that Submitted

Evaluations

Number of Documents Submitted

Formative SummativeCombination

Formative Summative

HEP 14 21 1 2 1

TYPE, continued(as defined by evaluator and/or noted on document)

Student/Staff Surveys Questionnaires Focus Groups Other (including site

visits, annual, etc.

0 4 2 11

29

V. Observations

30

General Observations

• Grantees determined the format for evaluation reports • All grantees may not have submitted all evaluation

reports on file in response to OME request• Although approved applications define grantee

evaluation processes, a number of submitted reports did not reference, or reflect fully, the processes indicated in the funded application

• Few reports addressed implementation of a planned budget

31

Observations: Component 1 – GPRA 1 and GPRA 2

• All HEP formative or summative evaluations provided Component 1 – GPRA 1 and GPRA 2 data

• Five HEP evaluation documents presented general information on validation of data for GRPA results

32

Observations: Component 2 – Fidelity of Implementation to Design

• Formative and summative evaluations presented data on progress in meeting GPRA 1 and 2 and measurable performance objectives

• Few evaluations presented data on grantee fidelity to implementation of design for instructional, support and placement services and management and recruitment plans

• Expanded information might have been provided through:– Documentation of services– Student and staff survey results– Tutoring and counseling logs– Class attendance sheets

33

Observations: Component 3 – Effectiveness of Project Design

• Instructional Services – HEP evaluations provided minimal data on instructional

services offered

• Support Services – References in several evaluations were limited to a

simple listing of the support services– One evaluation included detailed information regarding

delayed stipend payments, related staff interviews, and a proposed interim solutions

34

Observations: Component 3 – Effectiveness of Project Design, continued

• Management Plan– Limited discussion was included in reports about:• Management plan• Senior management• Administrative staff• Security and confidentiality measures for student records• Fiscal management

35

Observations: Component 3 – Effectiveness of Project Design, continued

• Recruitment Plan – To evaluate the effectiveness of the recruitment plan

design, few reports provided: • Demographic data• Selection and non-selection rates• Screening tools• References to trainings offered to recruiters

36

Observations: Component 4 – Collaborative Agreements

• Collaborative agreements key to helping grantees leverage Federal, State and local resources

• Ineffective execution of agreements can be a detriment to grantee success

• Few evaluations included discussion about the relationships or plans to expand collaborative partners

37

Observations: Component 5 – Recommendations

• Many evaluation documents did not provide extensive recommendations

• Recommendations often did not link directly to discussion within the report

• Five HEP evaluations made no recommendations• Texas A&M International University evaluation provided

comprehensive recommendations that were directly related to issues in the report

38

VI. Notable Evaluation Reports

39

Notable Evaluation Reports: Validation, Surveys

Reports notable for data validation, survey results and collaborative agreements

• Data Validation: – 2011-2012 Heritage University HEP

• Survey Results: – 2011 Texas State Technical College, Harlingen HEP

• Collaborative Agreements: – 2011 Texas State Technical College, Harlingen HEP

40

Notable Evaluation Reports: Overall

Notable Overall Evaluations:

• 2012 Hartnell Community College District, Hartnell College HEP

• 2009-2010 Texas A&M International University (TAMIU) HEP

• 2011 Texas State Technical College, Harlingen HEP

41

VII. Initial Grantee Feedback (HEP-CAMP ADM Session)

42

Initial Grantee Feedback

• Group 1: Effectiveness of Project Design - Umbrella – requires support from the following designs• Management plan• Recruitment plan• Support services• Placement services

– Instructional services design as the “handle” that supports the umbrella.

43


• Group 2: Rubric = Trash Receptacle• Repurpose the rubric by simplifying it• Most importantly, emphasize• Fidelity to your grant application and objectives• No. Served is important, fidelity to recruitment plan• GPRA 1 and GPRA 2, important to meet/exceed

44


• Group 3: Successful Evaluation is Dependent Upon• Data Validation• Data on Performance Measures (GPRAs)• Graduate HEP• Postsecondary, upgraded employment, military

45


• Group 4: HEP-CAMP Success = School – Takes nourishment and collaboration with • Agencies • K-12 MEP • Private Sector

46

VIII. Today’s Feedback

47

Today’s Feedback

Please consider the rubric, and provide feedback on the rubric.• What component(s) would you keep? Why?• What component(s) would you eliminate? Why?• What component(s) would you add? Why?

48

IX. Next Steps – Provide Continued Feedback

49

Recommendations for Grantee Evaluations

• In the future, OME may develop guidance/outline for evaluation reports

• Further grantee input is requested by e-mail to: [email protected]

• Resources may include:– HEP-CAMP Toolkit

http://www.hepcamptoolkit.org/hep/index.php/overseeing-your-grant/evaluation/

– MEP Evaluation Toolkit http://results.ed.gov/sites/results.ed.gov/files/pe-toolkit.pdf

50




http://results.ed.gov/sites/results.ed.gov/files/pe-toolkit.pdf



Rubric Components

51

OME HEP/CAMP RUBRIC for EVALUATION REPORTS

Grantee Name:

HEP or CAMP:

Evaluator Name/s:

Type of Evaluator: Internal; Third Party

Type of Report: Formative, Summative, Combination, Focus Group, Site Evaluation Form, Survey

52

GPRA Results

Component 1 GPRA Results

Evaluation Component

Does Not Meet Expectations Needs Improvement Meets

ExpectationsExceeds

Expectations Points

1. GPRA Results 0 1 2 3

1.a. GPRA 1 results The report does not include GPRA 1 results.

The report includes minimal information on GPRA 1 results.

The report presents GPRA 1 results and the

grantee’s attainment level

in relation to GPRA 1 target.

The report presents GPRA 1 results and a detailed description

of the grantee’s progress to meet

performance requirements.

x

1.b. GPRA 2 results The report does not include GPRA 2 results.

The report includes minimal information on GPRA 2 results.

The report presents GPRA 2 results and the

grantee’s attainment level

in relation to GPRA 1 target.

The report presents GPRA 2 results and a detailed description

of the grantee’s progress to meet

performance requirements.

x

53

GPRA Results, continued

Component 1 GPRA Results

Evaluation Component Does Not Meet Expectations

Needs Improvement Meets Expectations Exceeds Expectations Points

1. GPRA Results 0 1 2 3

1.c. The report presents data on the project

specific performance measures included in the

grantee’s application, including actual results

compared with targeted results.

The report presents no

information on other

performance measures.

The report only provides a listing

of other performance

measures.

The report includes data on most project specific performance measures included in

the grantee’s application

The report presents specific performance measures included in

the grantee’s application, including

actual results compared with

targeted results.

x

1.d. Data validation for GPRA 1 and

GPRA 2 data.

The reports presents no

information on validation of data.

The report provides minimal

reference to validation of

data.

The report discusses data validation

processes for the project.

The report discusses in detail the grantee’s

processes for validating the project

data.

x

1. Sub-Total 0

54

Fidelity of Implementation to Design

Component 2 Fidelity of Implementation to DesignInstructional Services



ExpectationsExceeds

Expectations Points

2. Fidelity of Implementation

to Design 0 1 2 3

2.a. Fidelity to instructional services design. The report presents qualitative and/or quantitative

data on the grantee's level of fidelity of

implementation to the instructional services design.

The report presents no qualitative and/or

quantitative data on the grantee’s level of

fidelity of implementation to the instructional services

design.

The report presents minimal qualitative and/or quantitative

data on the grantee’s level of fidelity of

implementation to the instructional services design.

The report presents

qualitative and/or quantitative data on the grantee’s

level of fidelity of implementation

to the instructional

services design.

The report presents qualitative and/or


fidelity of implementation to key elements in the

instructional services design.

x

55

Fidelity of Implementation to Design, continued

Component 2 Fidelity to Implementation of DesignSupport Services



ExpectationsExceeds

Expectations Points


to Design 0 1 2 3

2.b. Fidelity to support services

design. The report presents qualitative and/or quantitative


implementation to the support services

design.

The report presents no qualitative or

quantitative data on the grantee’s level of fidelity

of implementation to the support services

design.

The report presents minimal qualitative or quantitative data on the grantee’s level of

fidelity of implementation to

the support services design.

The report presents

qualitative or quantitative data on the grantee’s

level of fidelity of implementation to


The report presents qualitative and




x

56


Component 2 Fidelity of Implementation to DesignPlacement Services



ExpectationsExceeds

Expectations Points


to Design 0 1 2 3

2.c. Fidelity to the placement services design. The report

presents qualitative and/or quantitative

data on the grantee's level of

implementation to the placement services

design.



of implementation to the placement services

design.



the placement services design.

The report presents








x

57


Component 2 Fidelity of Implementation to DesignManagement Plan



ExpectationsExceeds

Expectations Points

2. Fidelity of Implementation to

Design 0 1 2 3

2.d. Fidelity to management plan design. The report

presents qualitative and/or quantitative


implementation to the management plan

design.



of implementation to the management plan

design.



the management plan design.

The report presents


level of fidelity of implementation to the management

plan design.




the management plan design.

x

58


Component 2 Fidelity to Implementation of DesignRecruitment Plan



ExpectationsExceeds

Expectations Points


to Design 0 1 2 3

2.e. Fidelity to the recruitment plan

design. The report presents qualitative and/or quantitative


implementation to the recruitment plan

design.



of implementation to the recruitment plan

design.



the recruitment plan design.

The report presents








x

2. Sub-Total 0

59

Effectiveness of Project Design

Component 3 Effectiveness of Project DesignInstructional Services


Does Not Meet Expectations

Needs Improvement

Meets Expectations

Exceeds Expectations Points

3. Effectiveness of Project Design 0 1 2 3

3.a. Instructional services design. The report analyzes the effectiveness of the

grantee's instructional services design.

Report does not include a

quantitative analysis of the impact of key

instructional services upon

GPRA results.

Report includes a quantitative analysis

of the impact of some instructional

services upon GPRA results.

Report includes a quantitative analysis of the impact of key

instructional services upon GPRA results.

Report includes a quantitative analysis of

the impact of all instructional services upon GPRA results.

x

3.a.2. The report includes data on the

type and usage of eligibility screening

tools.

Report does not include information

on tools for eligibility screening.

Report includes minimal data on the

type and usage of eligibility screening

tools and/or indicates an open enrollment

process.

Report includes general data on the type and usage of

eligibility screening tools and/or links an

open enrollment process to the grant

application.

Report specifies specific tools used for eligibility screening , procedures

and timelines for screening, and the

effect on performance.

x

60

Effectiveness of Project Design, continued

Component 3 Effectiveness of Project DesignInstructional Services



ExpectationsExceeds

Expectations Points


3.a.3. The report includes data on how instructional requirements of

individual students are assessed and met.

Report does not include data on individual

student instructional needs.

Report presents minimal data on

individual student instructional needs.

Report presents general data on

individual student instructional needs

and how the needs are

met.

Report provides extensive data on

assessment of individual student

instructional needs and the resources dedicated to meet

the identified needs.

x

61


Component 3 Effectiveness of Project DesignSupport and Placement Services



ExpectationsExceeds

Expectations Points


3.b. Support services design. The report

analyzes the effectiveness of the grantee's support services design.

No student survey data included.

Student survey data included, items align

minimally with services, and survey data

analyzed.

Student survey data included, items align mostly with services,

and survey data analyzed.


completely with services, and survey data is analyzed for effectiveness of the

support services.

x

3.c. Placement services design for HEP projects

No student survey data included.

Student survey data included,

items align minimally with grantees

placement services design plan.


partially with placement services

design plan.


completely with placement services

design plan.

x

62


Component 3 Effectiveness of Project DesignManagement Plan



ExpectationsExceeds

Expectations Points


3.d. Management plan design. The report

analyzes the effectiveness of the

grantee's management plan design.

No employee survey data included.

Employee survey data included, items align

minimally with management plan

design and survey data analyzed.

Employee survey data included, most

items align with

management plan design and survey

data analyzed.

Employee survey data included, items align

completely with management plan

design and survey data analyzed.

x

3.d.1. Teaching and administrative staff

structure.

The report presents no breakdown data on staffing structure.

The report presents minimal breakdown

data on staffing structure.

The report presents data on staffing

structure.

The report presents quantitative data

on staffing structure, including full-time

equivalents (FTEs) in each functional area.

x

63





ExpectationsExceeds

Expectations Points


3.d.2. Senior project administrative staff.

The report provides no description for senior administrative staff.

The report provides minimal discussion

about the senior administrative staff.

The report identifies the senior

administrative staff.

The report identifies the senior

administrative staff and their qualifications for

project roles.

x

3.d.3. Qualifications of project instructors.

The report presents no data on qualifications of

instructors.

The report present minimal data on qualifications of

instructors.

The report presents qualification data for

some instructors.

The report presents data on the

qualifications of all project instructors.

x

3.d.4. Professional staff development

The report presents no data on professional staff

development.

The report presents minimal data on professional staff

development.

The report provides data on some

professional staff development.

The report presents extensive data on professional staff

development.x

64





Needs Improvement

Meets Expectations Exceeds Expectations Points


3.d.5. Student recordsThe report provides no

qualitative or quantitative data on

student records.

The report provides minimal qualitative and/or quantitative

date on student records.

The report provides qualitative and/or

quantitative data on student records

examined.

The report provides extensive quantitative and

qualitative data on the completeness and

condition of student records examined.

x

3.d.6. Approved procedures policy

manual

The report provides no information about the

presence of a procedures policy manual.

The report indicates there is a procedures

policy manual.

The report indicates the presence of an

current policy manual.

The report evaluates the quality and utility of a

current policy procedures manual.

x

3.d.7. Financial/cost sharing issues with

host institution

The report presents no information on cost

sharing issues with the host institution.

The report presents brief discussion about

cost sharing issues with the host

institution.

The report discusses financial and cost

sharing issues with the host institution.

The report provides discussion of cost sharing issues and how to resolve.

x

65




Does Not Meet Expectations Needs Improvement Meets Expectations Exceeds Expectations Points


3.d.8. Student stipendsThe report

provided no data on stipends.

The report provided minimal data on

availability and plan for distribution of

stipends.

The report provided data on the type of

available stipends and distribution.

The report described in detail the stipends,

distribution schedules and any related issues.

x

3.d.9. Planned and actual cost


quantitative data on the planned and actual annual cost

per student

The report presents minimal quantitative data on the planned

and actual annual cost per student.

The report presents quantitative data on

the planned and actual annual cost per

student as well as detailed information about any carry-over

funds.

The report presents detailed quantitative data on the planned

and actual annual cost per student and

identified reasons for any variances as well as

detailed information about any carry-over

funds.

x

66


Component 3 Effectiveness of Project DesignRecruitment Plan


Does Not Meet Expectations Needs Improvement Meets Expectations Exceeds Expectations Points


3.e. Recruitment plan design. The report

analyzes the effectiveness of the

grantee's recruitment plan design.


information on the effectiveness of the

recruitment plan design to enroll

students.

The report provides minimal information

about the effectiveness of the

recruitment plan design to enroll

students.

The report presents some qualitative

information on the effectiveness of the recruitment plan for

students.

The report presents detailed qualitative information on the effectiveness of the

recruitment plan designed to enroll the

planned number of students.

x

3.e.1. Demographic information

The report provides no

demographic data on students recruited.

The report provides limited demographic data on the students

recruited to participate.

The report provides demographic data, including age and

gender of recruited students.

The report provides detailed demographic

data for recruited students by age and

gender and corresponding success

rates.

x

67


Component 3 Effectiveness of Project DesignRecruitment Plan



Needs Improvement



3.e.2. Training for Recruiters

The report does not include any training program for project

recruiters.

The report includes limited data on

electronic and/or in-person training

programs for project recruiters.

The report provides general information

on electronic and in-person training opportunities for recruiters, topics,

timelines, and completion rates.

The report includes detailed information on electronic

and in-person training opportunities for recruiters,

topics including outreach/enrollment techniques, timelines, completion rates, and

evaluation of contribution of trainings to recruiters'

success rates.

x

3. Sub-Total 0

68

Collaborative Agreements

Component 4 Collaborative AgreementsExisting and Planned




4. Collaborative Agreements 0 1 2 3

4.a. Existing collaborative agreements

The report provides no listing or analysis of

existing collaborative agreements with

educational institutions or service providers.

The report lists existing collaborative

agreements with educational

institutions or service providers.

The report provides some analysis of

existing collaborative agreements with


The report provides in-depth analysis of existing collaborative agreements

with educational institutions or service

providers.

x

4.b. Planned collaborative agreements

The report provides no analysis of planned

collaborative agreements with


The report provides names of planned

collaborative agreements with

educational institutions or

service providers.

The report provides analysis of some

planned collaborative agreements with


The report provides analysis of and reasons for additional planned

collaborative agreements with educational

institutions or service providers.

x

69

Collaborative Agreements, continued

Component 4 Collaborative AgreementsRelationships with Host Institutions



Needs Improvement


4. Collaborative Agreements 0 1 2 3

4.c Relationships with host institutions,

including unresolved facilities,

equipment/computers, HR and related

issues.

The report presents no qualitative

information on issues with host institutions that

affect project performance

The report presents minimal facilities,


issues with host institutions but

does not discuss impact on program

performance.

The report presents any facilities,


issues with host institutions and discusses impact

on program performance.

The report presents facilities,

equipment/computers, HR and related issues with host

institutions that impact program performance and how they can be resolved.

x

4. Sub-Total 0

70

Component Recommendations

Component 5 RecommendationsInstructional and Support Services




5. Component Recommendations 0 1 2 3

5.a. Instructional services


recommendations to improve GPRA 1 and

2 results.

The report presents minimal recommendations to improve GPRA 1

and 2 results.

The report provides some

recommendations to improve GPRA 1

and/or 2 results that are linked to report

information.

The report provides extensive

recommendations to improve GPRA 1 and 2 results that are linked to report information.

x

5.b. Support ServicesThe report presents no

recommendations for support services.

The report presents minimal recommendations

for improving support services.


recommendations for improving support services that are linked to report

information.


recommendations on improving support

services that are linked to report information.

x

71

Component Recommendations, continued

Component 5 RecommendationsPlacement Services and Management Plan





5.c. Placement services


recommendations for placement services.

The report presents minimal recommendations

on improving placement services.


recommendations for improving placement

services that are linked to report

information.


recommendations on improving placement

services that are linked to report information.

x

5.d. Management plan


recommendations for the management

plan.

The report presents minimal recommendations on improving the

management plan.


recommendations for improving the

management plan that are linked to

report information.


recommendations on improving the

management plan that are linked to report

information.

x

72

Collaborative Agreements, continued

Component 5 RecommendationsRecruitment Plan



Needs Improvement



5.e. Recruitment plan


recommendations for strengthening the recruitment

plan.

The report presents minimal

recommendations on strengthening the recruitment

plan for improved results.


recommendations for strengthening the recruitment plan results that

are linked to report information.

The report provides extensive recommendations

on strengthening the recruitment plan to improve

results that are linked to report information.

x

5. Sub-Total 0

TOTAL Rubric Score

73

the state of high school equivalent program (hep) evaluation u.s. department of education office of...

Documents

methods of evaluation

evaluation design

strong evaluation plan

funded project

project participants

specific project objectives

section d project goals

hep application section