document resume ed 382 802 ce 068 882 author stecher, … · document resume ed 382 802 ce 068 882...
TRANSCRIPT
DOCUMENT RESUME
ED 382 802 CE 068 882
AUTHOR Stecher, Brian M.; And OthersTITLE Improving Performance Measures and Standards for
Workforce Education.INSTITUTION National Center for Research in Vocational Education,
Berkeley, CA.SPONS AGENCY Office of Vocational. and Adult Education (ED),
Washington, DC.PUB DATE May 95CONTRACT V051A30003-95A; V051A30004-95ANOTE 65p.; For an earlier study, see ED 378 364.AVAILABLE FROM National Center for Research in Vocational Education,
Materials Distribution Service, Western IllinoisUniversity, 46 Horrabin Hall, Macomb, IL 61455 (orderno. MDS-821: $3.50).
PUB TYPE Reports Research/Technical (143)Legal /Legislative /Regulatory Materials (090)
EDRS PRICE MF01/PC03 Plus Postage.DESCRIPTORS *Accountability; *Competency Based Education;
Educational Planning; *Federal Legislation; JobTraining; Performance Factors; PostsecondaryEducation; Program Development; Program Evaluation;Program Implementation; *Program Improvement;Secondary Education; *Standards; TechnicalAssistance; Vocational Education
IDENTIFIERS *Carl D Perkins Voc and Appl Techn Educ Act 1990
ABSTRACT
This report extends the results of an earlier studyon the use of performance measures and standards to suggestprinciples for designing outcome-based program improvement systems.It examines the implications of that research for enhancingaccountability in future federal work force preparation legislation.It also illustrates specifically how the language of the Carl PerkinsVocational and Applied Technology Education Act of 1990 (Perkins II)could be changed to carry out the recommendations of the earlierstudy. Four features are identified that were lacking in Perkins IIbut are important for an outcome-based system to promote effectiveprogram improvement: (1) coordinate separate components into a moreintegrated system for planning, implementing, monitoring, andimproving vocational education and training; (2) increase theemphasis on the use of the system of performance measures andstandards as a program improvement tool; (3) clarify the requirementsfor measures and standards and improve their technical quality; and(4) increase the amount of technical assistance provided by federaland state agencies to support change at the local and state levels.Specific examples are given of changes in the language of Perkins II
to incorporate these principles. An appendix contains the completetext of the proposed revisions with detailed commentary comparing thenew law to the old. (YLB)
1
I ()
2BEST COPY AVAILABLE
National Center fork:search inVocational Education
University of California. Berkeley
IMPROVING PERFORMANCEMEASURES AND STANDARDS
FOR WORKFORCE EDUCATION
U.E. DEPARTMENT OF EDUCATIONOffice of Educator* Research and Improvement
EDUCATIONAL. RESOURCES INFORMATIONCENTER IERIC)
,0/tus document has been reproduced asreceived from the pinion or organizationorerneting It
0 Minor changes have been made to improvereproduction Quality
Points of mow of opinions stated in this docu-ment do not netessetnty represent officialOf RI paean or policy
Supported bythe Office of Vocational and Adult Education,
U.S. Department of Education
This publication is available from the:
Nation& Center for Research in Vocational EducationMaterials Distribution ServiceWestern Minois University46 Horrabin HallMacomb, IL 61455
800-637-7652 (Toll Free)
BEST COPYAVAILABLE 3
IMPROVING PERFORMANCEMEASURES AND STANDARDS
FOR WORKFORCE EDUCATION
Brian M. StecherLawrence M. Hanser
RAND
Mika la L. RahnKaren LevesqueSteven G. KleinDavid Emanual
MPR Associates
National Center for Research in Vocational EducationGraduate School of Education
University of California at Berkeley2150 Shattuck Avenue, Suite 1250
Berkeley, CA 94704
Supported byThe Office of Vocational and Adult Education,
U.S. Department of Education
May, 1995 MDS-821
Project Title:
Grant Number:
Act under whichFunds Administered:
Source of Grant:
Grantee:
Director.
Percent of Total GrantFinanced by Federal Money:
Dollar Amount ofFederal Funds for Grant:
Disclaimer:
Discrimination:
FUNDING INFORMATION
National Center for Research in Vocational Education
V051A30003-95A/V051A30004-95A
Carl IX Perkins Vocational Education ActP.L. 98-524
Office of Vocational and Adult EducationU.S. Department of EducationWashington, DC 20202
The Regents of the University of Californiac/o National Center for Research in Vocational Education2150 Shattuck Avenue, Suite 1250Berkeley, CA 94704
David Stern
100%
$6,000,000
This publication was prepared pursuant to a grant with the Officeof Vocational and Adult Education, U.S. Department ofEducation. Grantees undertaking such projects undergovernment sponsorship are encouraged to express freely theirjudgement in professional and technical matters. Points of viewof opinions do not, therefore, necessarily rerresent official U.S.Department of Education position or policy.
Title VI of the Civil Rights Act of 1964 states: "No person inthe United States shall, on the ground of race, color, or nationalorigin, be excluded from participation in, be denied the benefitsof, or be subjected to discrimination under any program oractivity receiving federal financial assistance." Title IX of theEducation Amendments of 1972 states: "No person in theUnited States shall, on the basis of sex, be excluded fromparticipation in, be denied the benefits of, or be subjected todiscrimination under any education program or activity receivingfederal financial assistance." Therefore, the National Center forResearch in Vocational Education project, like every program oractivity receiving financial assistance from the U.S. Departmentof Education, must be operated in compliance with these laws.
NCRVE, MDS-821
ACKNOWLEDGMENTS
The research which informed this analysis could not have been conducted without
the assistance of secondary and postsecondary vocational educators in seven states, who
gave generously of their time and insights. Our colleagues E. Gareth Hoachlander and
Bryan Hallmark also were important contributors to this study. Judy Wood assisted with
.ext editing and document preparation.
i 6
NCRVE, MDS-821
PREFACE
This report extends the results of an earlier NCRVE study, Improving Perkins II
Performance Measures and Standards: Lessons Learned from Early Implementers in Four
States, published in 1994, on the use of performance measures and standards to suggest
principles for designing outcome-based program improvementsystems_ialight of current
efforts to reform the U.S. workforce education system. At the time this project was
undertaken, states were actively involved in implementing the Carl D. Perkins Applied
Technology and Vocational Education Act of 1990 (Perkins II), and educators andpolicymakers were anticipating the reauthorization of the legislation in 1995. However, the
policy environment has changed; other options such as consolidated block grants for
education and training are receiving increasing attention. Although in its details the present
document reflects an emphasis on Perkins II, in its general principles the report should be
of interest to federal policymakers engaged in developing new education and training policy
initiatives. A companion RAND Issue Paper, Accountability and Workforce Training
(Stecher & Hanser, 1995), discusses the implications of this study and related research on
accountzbility in a non-Perkins environment.
NCRVE, MDS-821
SUMMARY
The Carl D. Perkins Vocational and Applied Technology Education Act of 1990
(Perkins II) has guided federal vocational education policy for the past five years. One of
the most significant components of Perkins II was its emphasis on using systematic
outcome data as a program monitoring and improvement tool. A recent NCRVE study of
the effects of Perkins II, Improving Perkins II Performance Measures and Standards:
Lessons Learned from Early Implementers in Four States (Stecher et al., 1994), found that
the performance measures and standards provisions designed to promote programimprovement were not achieving their full potential; it identified shortcomings and
recommended actions that could be taken to improve the act.
This report examines the implications of that research for enhancing accountability
in future federal workforce preparation legislation. It also illustrates specifically how the
language of Perkins II could be changed to carry out the recommendations of the earlier
study.
The following four features were identified as lacking in Perkins H but were found
to be important for an outcome-based system to promote effective program improvement:
1. Coordinate separate components into a more integrated system for planning,
implementing, monitoring, and improving vocational education and training.
2. Increase the emphasis on the use of the system of performance measures and
standards as a program improvement tool.
3. Clarify the requirements for measures and standards and improve their technical
quality.
4. Increase the amount of technical assistance provided by state and federal agencies to
support change at the local and state levels.
Specific examples are given of changes in the language of Perkins II to incorporate
these principles.
TABLE OF CONTENTS
Acknowledgments i
Preface iii
Summary v
Introduction 1
Background 2
NCRVE Study of the Implementation and Impact of Measures and Standards 3
Translating Results Into Legislation 6
Rationale for Legislative Changes 7
Developing a Coordinated Program Improvement System 8
Use of Information for Program Improvement 10
Improved Measures and Standards 11
Technical Assistance 13
Proposed 1995 Perkins Act Sections 14
References 15
Technical Appendix 27
NCRVE, MDS-821
INTRODUCTION
Current federal vocational education legislation expires in 1995, presenting the
104th Congress with an opportunity to reshape federal policy regarding secondary and
postsecondary vocational education. At the time of this writing, it remains uncertainwhether the Carl D. Perkins Vocational and Applied Technology Education Act (Perkins
II) will be reauthorized in its present form or whether federal vocational education
initiatives will be "merged into a broader workforce-development bill" (Sommerfeld,
1994, p. 18). Given the growing emphasis on measurable outcomes and standards in
government programs, requirements for a system of outcome measures and standards are
likely to be included in new legislation, regardless of its format.
A recent study by the National Center for Research in Vocational Education(NCRVE), Improving Perkins II Performance Measures and Standards: Lessons
Learned from Early Implementers in Four States (Stecher et al., 1994), suggests ways to
enhance accountability in future federal vocational education legislation. This paper
reviews the findings and recommendations of that study and illustrates how they could be
translated into legislation. Specifically, we offer suggestions for more effective outcome
based program improvement and accountability procedures. The paper focuses only on
those provisions that relate to evaluation and program improvementoften discussedunder the heading "performance measures and standards." These suggestions are
presented as revisions to the existing Perkins legislation, but they are equally relevant in
the context of broader workforce development legislation. Therefore, the report should
be of general interest to poiicymakers responsible for workforce training at the state and
federal levels.
Examples of changes in language we suggest in this report are less relevant than
the principles that guided them. Specifically, the results of the study suggest that any
federal vocational-education effort should incorporate the following:
continued development of outcome-based program assessment tools
greater coordination between vocational education and workforce development
initiatives
NCRVE, MDS-821
increased emphasis on the use of performance information for programimprovement
increased technical assistance to support these objectives
Furthermore, the language presented in this document may provide a useful starting point
for other conceptualizations-of theederal role in vocational education.
Background
Perkins II has guided federal vocational education policy for the past five years.
It has been a remarkably influential piece of legislation, in part. because most states have
opted to apply its provisions more broadly than required. Although the federalgovernment supplies less than ten percent of the total resources devoted to secondary and
postsecondary vocational education, most states have applied Perkins II requirements to
locally and state-funded vocational education efforts as well as to those funded with
federal resources s.
One of the most significant changes embodied in Perkins II was an emphasis on
using systematic outcome data as a monitoring and improvement tool for programs.Under Perkins II, states were required to develop statewide "systems of core standards
and measures of performance" that would be used to determine the success of vocational
programs and to serve as a basis for local program improvement. If necessary, these
systems could also be used to justify state intervention. States were given considerable
flexibility in developing their statewide systems; the law specified only two outcomes
that were required to be measured. At a minimum, each state was required to collect
measures of learning and competency gains in basic and more advanced academic skills,
as well as at least one measure of occupational competency attainment, job or work skill
attainment, or retention or-student-placeMent. Any additional measured outcomes were at
the discretion of the Committees of Practitioners and state Departments of Education.
Those who endorsed this approach to program monitoring in vocational education
hoped it would lead to better evaluation, richer communication, more focused program
improvement at the local level, and wiser use of state technical assistance capabilities. It
was hoped that states would implement efficient systems that provided local
2 1 1
I
tl
1
NCRVE, MDS-821
administrators and instructors with meaningful performance data to assess the strengths
and weaknesses of their vocational programs and to design new strategies to improve the
academic, technical, and labor-market outcomes of their students.
These hopes have not been fully realized, although states have made substantial
strides in implementing performancc measures and standards systems as envisioned in the
federal legislation. However, there remains considerable room for improvement. The
following summarizes the results of a recent NCRVE study on the implementation and
impact of the Perkins II measures and standards. These findings form the basis forsuggested changes in the legislation, which are elaborated in the section entitled"Rationale for Legislative Changes."
NCRVE Study of the Implementation and Impact of Measures and Standards
In the spring of 1993, NCRVE initiated a two-year study of the effects of Perkins
II performance measures and standards for vocational education. At the time of the
study, states had three years to implement these provisions. The study examined seven
states' progress in implementing statewide systems Gf performance measures andstandards, the effects of these systems on local vocational programs and state agencies,
and the factors that influenced local and state actions.'
Four states which were "early adopters" of measures and standards were initially
selected for study. In each state, we interviewed staff in the state agency (or agencies)
that administered secondary and postsecondary vocational education, and administrators
and instructors in both a secondary and a postsecondary vocational institution in twogeographically separated regions. Respondents were asked about a number of related
themes, including the implementation of performance measures and standards, theirintegration with other educational reform efforts, and the impact of measures and
standards on their vocational programs. Repeat visits to each vocational institution were
conducted the following year. In the second year, three additional states were added to
the sample to provide greater contrast in terms of implementation and to broaden our
exposure.
I No formal attempt was made to determine whether states were in compliance with Perkins II or to judgethe quality of the measures and standards states had chosen to implement.
312
NCRVE, MDS-821
Results
This section summarizes the findings of the NCRVE study (Stecher et al., 1994).
Substantial progress had been made in implementing measures and standards in the states
that were visited, although much work remains to be done to make the systems function
as envisioned in the law. At the time of our visits, little attention had been paid tobuilding local- or state-level capacity for translating the measures_ and standards data into
actions for local program improvement. These "leading edge" states were still largely
engaged in developing and implementing their systems.
Furthermore, large variation was found in the states' approaches to thedevelopment and implementation of measures and standards. This variation was evident
in almost every aspect of program implementation, including how the process wasmanaged, who participated. and the level of resources devoted to it. These differences
appeared to be jointly a function of the states' individuality and the flexibility inherent in
Perkins H.
We identified several factors that affected implementation and contributed to the
variation in state responses to performance measures and standards.2 Some of these
explanatory factors were elements of the local and state context, and are less responsive
to federal policy intervention. Other factors are within the sphere of federal policy
influence.
The level of flexibility afforded states was the first of five factors that couldclearly be affected by federal actions. On the positive side, flexibility permitted states to
create systems that were responsive to local conditions. On the negative side, the latitude
afforded states increased the influence of state context, which heightened differences
between states, and, in some cases, lengthened the implementation process.
The second explanatory factor was the separate and uncoordinated nature of the
elements of Perkins II. The Perkins LE prioritiesmeasures and standards, integration,Tech Prep, and service to special populationswere not seen as a coordinated system at
either the local or state levels. Similarly, performance measures and standards were not
being used comprehensively to evaluate the other Perkins initiatives.
2 See Stecher et al. (1994) for a complete description of the findings.
4
13
NCRVE, MDS-821
Third, there were neither models nor incentives for ensuring that performance
measures and standards were used to improve programs. Perkins II contained an explicit
framework ztructuring systems, and a federal agency checked for compliance at the
adoption stage. However, the law and regulations did little to emphasize information
use.3
The fourth factor concerned resources and expertise at the state level. Perkins II
created new responsibilities for state staff, but reduced the set-aside for stateadmInistration and provided little technical assistance. This presented a dilemma for
states that lacked either the expertise or the resources to address these new demands.
Finally, the law mandated measurement of learning outcomes, even though there
were few valid tests available for this purpose. The scarcity of appropriate tools formeasuring selected learning outcomes led states to adopt alternatives that were less than
optimal. States are still struggling with how to measure important student outcomes such
as academic skill gains at the postsecondary level.
RecommendationsThe study recommended several actions federal policymakers could take to
enhance the future success of performance measures and standards in vocationaleducation and to promote the program improvement goals of Perkins IL
Clarify the Interrelationships and Coordination Among Federal Mandates.
Policymakers should clarify the relationships among systems of measures and
standards, the integration of vocational and academic education, Tech Prepprograms, and service to special populations, as well as offer additional guidance
about coordinating states' efforts in these areas. For example, performance
measures and standards are clearly relevant to evaluating and improving School-
to-Work transition- programs, and the links between these activities should be
stressed in any future omnibus workforce legislation.
3 Some state vocational educators told us that Perkins 11 could be improved by adding "real" incentives andsanctions, that is, by tying funding to program performance. For example, they said that "joint planning" isnot an adequate penalty for continued failure to make substantial progress toward meeting standards.Termination of funding might be more appropriate. This issue created controversy at the time the law waswritten, and it continues to generate debate today. This study did not provide adequate evidence to supportor reject such a recommendation.
5 j4
NCRVE, MDS-821
Create Models for Outcome-Based Program Improvement.
At present, most state action is still driven by the mandate to develop the structure
of measures and standards. Little has been done to use that structure to make
programs better. Local and state agencies need assistance in translating outcome
deficiencies into action plans.
Provide Focused Technical Assistance for Choices and Resources.
The "flexible mandates" of Perkins II place greater demands on state agencies
while restricting the use of funds for state-level services. Federal actions that help
states respond to their choices and make better use of resources mightsignificantly improve Perkins II implementation. States need resources andexpertise to operate a reformed vocational education or workforce preparation
system, and the law should promote both.
Address Common Measurement Problems.
The technology to measure learning and occupational performance gains inreliable, valid, and efficient ways is not widely available. Most states are not
equipped with either the resources or expertise to fill this gap. The federalgovernment needs to assume leadership in addressing these problems, since they
are best solved nationally and are largely the result of federal legislationprovisions.
Translating Results Into Legislation
Incorporating these changes into the federal vocational education or workforce
preparation legislation will increase the efficacy of statewide systems of measures and
standards. While the results of this study do not speak to all aspects of the legislation,
they do suggest to us specific changes in the measures and standards provisions. The
section in this document ..IititlecP`Rationale for Legislative Changes" presents our
rationale for translating the findings of the study into changes in legislation. It elaborates
on four main themes drawn from the results, and presents specific recommendations for
revising the law. The next section, "Proposed 1995 Perkins Act Sections," contains a
marked-up version of Perkins II showing where the changes might be made in the event
of reauthorization. The paper concludes with a Technical Appendix that contrasts the
new and old language along with a commentary on proposed changes.
61t
NCRVE, MDS-821
We remind the reader that we have not attempted to rewrite the whole act or todraft omnibus legislation. We aim only to demonstrate how results of our research have
practical value in informing future legislation. Many of our suggested revisions toPerkins II could as easily be imbedded in a consolidated workforce preparation bill.
RATIONALE FOR LEGISLATIVE CHANGES
The changes we propose flow directly from the research findings summarized in
the previous section. In the case of performance measures and standards, the links
between federal legislation and state actions are relatively clear and direct, and it ispossible to trace implementation effects back to legislative causes. This linkagefacilitates the task of revising the law to promote desired outcomes. We believe that the
intent of the federal legislation was to promote effective program improvement, and that
this can best be accomplished by including four major changes in future legislation:
1. Coordinate separate components into a more integrated system for planning,implementing, monitoring, and improving vocational education and training.
2. Increase the emphasis on the use of the system of performance measures and
standards as a program improvement tool.
3. Clarif} and improve language describing the required measures and standards
themselves.
4. Increase the amount of technical assistance provided by state and federal agencies
to support change at the local and state levels.
To illustrate how these changes could be imbedded in federal legislation, we
revised selected portionso f- Sections 115, 116, and 117 of Perkins IIthe legislationdescribing performance measures and standards and the requirements for local and state
assessment and evaluation. The proposed revisions have been italicized. Data limitations
prevented us from undertaking a complete redrafting of the law. We have, therefore,
limited our efforts to only those sections where our earlier findings justify legislative
reworking. Readers should not assume that we endorse all the non-italicized sections of
NCRVE, MDS-821
the act; in most cases, these components are left unchanged because they were notinformed by our research.
The rest of this section describes in narrative format the major changes werecommend. This approach communicates better the goals we were trying to achieve and
the broad changes we made to achieve them. The complete text of the proposedrevisions, with detailed commentary comparing the new law to the old, is contained in the
Technica! Appendix.
Developing a Coordinated Program Improvement System
Our revisions attempt to coordinate the separate elements found in Perkins II into
a more integrated system for planning, implementation, monitoring, and improving
vocational programs. The logical model underlying this system is illustrated in Figure 1.
Revised language clarifies the interrelationships between the elements of Perkins II,including state needs assessments, measures and standards, annual local evaluations, and
program improvement plans. Our revisions also promote greater coordination ofmeasures and standards with other federal workforce and education initiatives in the
following ways:
Performance measures and standards are conceptualized as one part of a larger,
interconnected system for improving vocatiu.ial education at the local and state
levels. The system includes many components, all of v. hich interact and inform
one another. For example, the state needs assessment drives the system ofmeasures and standards and the local application process. These, in turn, inf
the annual local evaluation, which leads to program improvement planning, to
periodic review of the system of measures and standards, and ultimately back to
the state needs assessment. In this way the information comes full circle and
helps to promote improvement of the whole system.
To make performance measures and standards a more comprehensive program
improvement tool, they are applied to all major Perkins instructional initiatives
(including programs funded under Title III such as Tech Prep and consumer and
home economics), not just to programs receiving funds under Title II.
8 1'1
NCRVE, MDS-821
The Committee of Practitioners is required to review and revise the measures and
standards annually, creating a self-improving system that is more responsive to
changes in other vocational education policies.
Secretaryof
Education
Figure 1
Proposed Legislative Relationships
:.CC? T.
LocalRecipients
Sec. 119Provide technical
assistance
Sec. 115Develop and implement a
system for improvingvocational education
16.Sec. 116
Conduct a state needsassessment
StatePlan
E.. LocalApplication
Sec. 117Revise core performancemeasures and standards
Sec. 120Evaluate state andnational standards
Sec. 119Provide technical
assistance
Sec. 117Collect data for
performance measuresand standards
Sec. 118Annual evaluation
In our research, we recognized the clear need for coordination among federal
workforce preparation programs. As a result, our suggested revisions in this area are
particularly relevant in the context of. a consolidated bill. Coordination among federal
workforce education programs is promoted in the following ways:
The State Board and the Committee of Practitioners are directed to consider all
federal training and workforce preparation efforts when developing measures and
standards and the program improvement system. Representatives of the agencies
BEST COPY AVAILABLE
918
NCRVE, MDS-821
responsible for administering related programs must be consulted in theappointment of the Committee of Practitioners.
The State Board is directed to provide the Committee of Practitioners withinformation about other state assessment efforts that might be relevant to acomprehensive program improvement system.
The federal government is directed to provide technical assistance to help states
coordinate measures and standards with other federal initiatives.
Use of Information for Program Improvement
Our proposed changes represent a significant change in focus away from the
initial development of the systems of measures and standards called for in Perkins II
toward the use of these systems for future program improvement. Our research revealed
that while states had made significant progress in developing their systems of measures
and standards, for the most part they had not yet tackled the next step of using their
systems for program improvement. Without explicit provisions for the use ofperformance measures and standards, the data they generate may languish in government
files instead of being used to improve programs. Although the theme of measures and
standards as the basis for a system of program improvement runs throughout ourproposed revisions, it is most evident in Section 118, which sets specific requirements for
program evaluation and improvement.
Increased relevance and usefulness of the annual evaluations for programimprovement is promoted in the following ways:
The requirement for program evaluation is explicitly written to include all the
relevant component programs of Perkins (e.g., Title III programs such as Tech
Prep), not just those efforts funded under Title II. This change ensures that
information about which components of Perkins are or are not working is
available to assist in program improvement planning.
Teachers and parents are explicitly included in the evaluation process because
they are important stakeholders and can contribute to the improvement of
programs.
10
19
NCRVE, MDS-821
The performance of special populations enrolled in vocational programs is to be
specifically compared to the performance of other vocational students to ensure
that staff monitor the access and success of students from special populations in
each program area.
Additionally, program improvement activities are promoted in the followingways:
The outcome of the annual evaluation is tied explicitly to a plan of programimprovement emphasizing a link that is not sufficiently clear in currentlegislation.
To promote program improvement as a continuing and ongoing process, arequirement has been added for all recipients to develop a local improvement
plan, regardless of whether standards have been met.
A requirement that the local improvement plan include a summary of the results
of the local annual evaluation strengthens and emphasizes the link between these
activities.
A requirement that those local recipients who fail to meet their standards describe
"specific strategies for making substantial progress" encourages follow-through
from evaluation to program improvement.
Improved Measures and Standards
Measures and standards are the cornerstone of the Perkins program improvement
system. Our revisions attempt to improve measures and standards in two ways: (1) by
clarifying the requirements for such systems and (2) by improving the technical quality of
the components. Clarification is achieved by adding definitions, reformulatingdescriptions, and reorganizing state options to reveal more of the existing similarities.
Technical quality is enhanced by requiring that states pay more attention to questions of
reliability, validity, and lack of bias, and by requiring that the federal government
contribute to the solution of common measurement problems.
NCRVE, MSS -821
Clarification of systems of measures and standards is achieved in the followingways:
Our revisions provide definitions for key terms that should be commonthroughout the country, including outcome, measure, standard, program, and
special populations. Because of differing local contexts, we feel the definition of
"substantial progress" ought to be a state prerogative, so the regulations require
states to define this term in a measurable and consistent manner.
The requirements for measures and standards are reformulated and reorganized to
make them more logical and consistent. State systems must contain at least five
measures (most already contain more than this), one eacn from the followingcategories: learning gains, work-related skills, retention or completion,placement, and access/equity. This ensures that prop ms are evaluated based
upon a complete and balanced picture of their desired effects.
Technical quality is promoted in the following ways:
States are required to revise measures and standards regularly, based on anexamination of technical quality and usefulness. This ensures that important
policy decisions are based on sound information.
The federal government is required to contribute to the solution of commonmeasurement problems such as evaluating the reliability, validity, and lack of bias
in measures; measuring academic gains at the postsecondary level; and setting
valid performance standards. These common problems, which were created by
the legislation, are shared by all states and are most efficiently addressednationally.
To focus attention on the goal of program improvement and the difficult problems
of measures and standards; the Secretary is required to report to Congress on the
status of each state's system for improving vocational education and on thetechnical quality of the measures and standards that are adopted.
12
NCRVE, MDS-821
Technical Assistance
Several activities required under Perkins II were beyond the capacity of the local
and state recipients to accomplish on their own. As a result, we have included a new
Section 119, titled "Technical Assistance," that specifies actions the state and federal
governments will take to assist recipients in carrying out their responsibilities forprogram improvement under reauthorized or consolidated legislation. We proposefundamentally different roles for the state and federal governments in the provision of
technical assistance. Our distinction is based on those problems that we perceive as best
solved nationally versus those that are prima..ily local and state in nature. The technical
assistance role we propose for the federal government includes the following:
Assistance to help states address measurement-related issues, includingestablishing the technical quality of measures (i.e., reliability, validity, and lack of
bias). This category also includes assistance in developing acceptable assessment
tools.
Assistance with identifying, developing, and implementing program improvement
strategies on a national level, in particular, helping to disseminate more broadly
program improvement models and other lessons learned in individual states.
Assistance with coordinating the measure...) and standards requirements with other
evolving and emerging workforce enhancement initiatives, including industry
skill standards and School-to-Work performance standards. It is our belief that
coordination will increase the effectiveness of these efforts.
The states have the essential role of assisting local recipients in evaluating and
improving their programs. This role includes the following responsibilities:
Assistance in evaluating the effectiveness of programs based on performance
measures and standards.
Assistance in identifying and adopting appropriate strategies for improving
performance.
Training local administrators and instructors in using performance data toimprove vocational programs and courses.
13 2 2
NCRVE, MDS -821
Dissemination to local recipients of examples of effective performance rata use to
improve vocational courses and programs.
PROPOSED 1995 PERKINS ACT SECTIONS
This section illustrates how the recommendations above could be translated into
specific legislation. We present the reforms in the context of Perkins II because that was
the relevant federal law at the time of this study. Similar wording would be appropriate
for achieving similar goals in the context of a consolidated approach to workforcetraining. The changes are highlighted in italics in the following text. We did not attempt
to rewrite the entire act, just to revise those sections that relate to program improvement
and accountability.
SEC. 115. IMPROVING VOCATIONAL EDUCATION AT THE STATE AND
LOCAL LEVEL
(a) GENERAL AUTHORITYEach State Board receiving funds under this Actshall develop and implement a system for improving vocational education at the
State and local level that includes:
(1) reappointing a State Committee of Practitioners (Section 115);
(2) conducting a State needs assessment in the first year underreauthorization and every two years thereafter (Section 116);
(3) implementing a set of core performance measures and standards (Section
117);
(4) condtc.ting annual evaluations of program effectiveness (Section 118);
and
(5) providing technical assistance to local recipients on using performance
data to improve vocational courses and programs (Section 119).
This system shall build on the system initiated under the 1990 Perkins Act, and
apply to all programs (as defined in Subsection (c)) receiving funds under Title II,
and as appropriate, to programs funded under Title III of this Act.
(b) REQUIREMENTEach State Board, before the expiration of the 30-day period
beginning on the date of the enactment of the 1995 Perkins Act, shall reappoint a
Ptate Committee of Practitioners (referred to as the "Committee") to develop the
14 23
NCRVE, MDS-821
elements of a program improvement system contained under the above general
authority. Committee members will be appointed in consultation with local
school officials representing eligible recipients, and representatives of organized
labor, business, superintendents, community-based organizations, private industry
councils established under Section 102(a) of the Job Tranling Partnership Act,
State councils, parents, special populations, correctional institutions, theadministrator appointed under Section 111(b)(1), the State administrator ofprograms assisted under Part B of the Education of the Handicapped Act, theState administrator of programs assisted ander Chapter 1 of Title I of theElementary and Secondary Education Act, the State administrator of programs for
students of limited English proficiency, and guidance counselors, andadditionallythe State administrator of programs developed under Title 11,Subtitle A of the School-to-Work Opportunities Act of 1994, and the Stateadministrator of programs assisted under Title III of the Goals 2000: Educate
America Act.
(c) DEFINITIONSThe State Board shall adopt the following definitions in itsState plan:
(1) Outcomea measurable characteristic of student or programperformance;
(2) Measurea method for quantifying performance in an outcome area;
(3) Standardthe desired level of attainment on a measure, or the rate ofprogress on a measure;
(4) Programa sequence of courses or instruction in a sequence oraggregation of occupational competencies that includes occupationallyspecific skills, general employability skills, work attitudes, appliedacademic learning, higher-order reasoning, and problem-solving skills;
(5) Special populationsincludes individuals with disabilities, educationally
and economically disadvantaged individuals (including foster children),
individuals of limited English proficiency, individuals in programsnontraditional for their sex, and individuals in correctional institutions.
The State Board shall also define the term "substantial progress" in its State plan
so that the term is measurable and applies consistently to all recipients of Perkins
funds in the state.
NCRVE, MDS-821
SEC. 116. STATE NEEDS ASSESSMENT
(a) IN GENERALEach State Board receiving assistance under this Act shallconduct an assessment using measurable objective criteria developed by tree State
Board to assess program quality. Such criteria shall be developed in consultation
with representatives of the Committee described in Section 115(b) and shall use
information gathered by the National Occupational Information Coordinating
Committee and, as appropriate, other information. Each State Board shall widely
disseminate such criteria. State Boards shall develop such criteria no later than
the beginning of the 1996-1997 school year. Such criteria shall include
( 1) Information on outcomes
(A) .student gains in and/or attainment of academic, occupational,
and/or generic work skills;
(B) student retention in and/or completion of secondary orpostsecondary education and/or vocational programs;
(C) student placement into additional training or education, military
service, and employment;
(D) other outcome areas identified by the State under Section 115 of
the 1990 Perkins Act;
(2) Information on instructional practices and experiences
(A) integration of academic and vocational education;
(B) the ability of the eligible recipients to meet the needs of special
populations with respect to vocational education, includingproviding incentives for participation, retention, and completion;
(C) assessing the quality of vocational education programs in schools
with high concentrations of poor or low-achieving students;
(D) enrollment retention, and completion rates of students invocational programs nontraditional for their sex;
(E) sequential courses of study leading to both academic andoccupational competencies;
(F) linkages between secondary and postsecondary educationalinstitutions;
(G) instruction and experience, to the extent practicable, in all aspects
of the industry the students are preparing to enter,
(H) the relevance of programs to the workplace and to the occupations
for which students are to be trained, and the extent to which such
16 25
NCRVE, MDS-821
programs reflect a realistic assessment of current and future labor
market needs, including needs in areas of emerging technologies;
(I) the ability of the vocational curriculum, equipment, andinstructional materials tt) meet the demands of the workplace;
(J) basic and higher order current afif_i_ future workplace competencies
that will reflect the hiring needs of employers;
(K) other factors considered appropriate by the State Board; and
(3) Procedures for using existing resources and methods developed in other
programs receiving Federal assistance.
(b) DEADLINE FOR ASSESSMENTEach State Board shall complete theassessment required by Subsection (a) before the expiration of the 6-month period
beginning on the date of the enactment of the 1995 Perkins Act, and every two
years thereafter.
SEC. 117. STATE AND LOCAL MEASURES AND STANDARDS
(a) GENERAL AUTHORITYEach State Board receiving funds under this Act
shall revise and implement a set of core standards and measures of performance
for secondary and postsecondary vocational education programs. To advise the
State Board, the Committee shall
(1) evaluate the reliability, validity, and lack of bias of the measures and
standards developed under Section 115 of the 1990 Perkins Act;
(2) assess the extent to which data generated under Section 115 of the 1990
Perkins Act were used effectively by recipients to improve programs and
identify barriers to effective use;
(3) recommend revisions to existing measures and standards; and
(4) develop new measures and standards as required under Subsection (b).
Revisions to the core measures and standards will be recommended by theCommittee and approved by each. State Board no later than one year after the
enactment of the 1995 Perkins Act. This core shall apply to all programs (as
defined in Section 115(c)(4)) receiving funds under Title II of this Act.Additionally, the State Board may require that the core apply to any programs
funded under Title III such as consumer and homemaking education and tech-
prep. Eligible recipients may make local modifications to such core based on
economic, geographic, or demographic factors, or the characterisiics of the
NCRVE, MDS-821
population to be served. Such modifications shall conform to the assessmentcriteria contained in the State plan. The State Board shall convene the Committee
on a regular basis to review, comment on, and propose revisions to the State's
core standards and measures of performance for vocational programs.
(b) REQUIREMENTSEach system developed under Subsection (a) shall include
at least five measures and standards
( I) a measure of student learning gains in the achievement of academic skills;
(2) 1 or more of the following measures of work-related skills
(A) occupational competency attainment in the industry a student is
preparing to enter;
(B) attainment of work readiness or generic work skills;
(C) attainment of understanding of all aspects of the industry a student
is preparing to enter;
(3) 1 or both of the following measures of retention or completion
(A)
(B)
retention in or completion of secondaryeducation;
retention in or completion of secondary
or postsecondary
or postsecondaryoccupational programs or other programs funded under Title III,
as appropriate;
(4) a measure of placement into additional training or education (including
placement into the postsecondary component of tech-prep or other School-
to-Work transition program), military service, or employment; and
(5) a measure of access and equity such as rates of enrollment in, retention in,
and/or completion of occupational or other programs for specialpopulations in comparison to non-special populations, including male and
female students in programs nontraditional for their sex.
(c) CONSISTENCY AND COORDINATION WITH OTHER PROGRAMSIn
developing the standards and measures included in a system developed under
Subsection (a), the State Board shall
(1) consider standards and measures developed under job opportunities and
basic skills training programs established and operated under a planapproved by the Secretary of Health and Human Services that meets the
requirements of Section 402(a)(19) of the Social Security Act;
18
NCRVE, MDS-82!
(2) consider standards prescribed by the Secretary of Labor under Section 106
of the Job Training Partnership Act;
(3) coordinate with State efforts to implement performance measuresdeveloped under Title IV, Section 402 of the School-to-Work Opportunities
Act;
(4) build on the content and student. performance_ standards. and related
assessments developed under Title II, Part B of the Goals 2000: Educate
America Act in assessing student learning gains in the achievement ofacademic skills at the secondary level; and
(5) build on the industry skill standards and related assessments andcertifications developed under Title V of the Goals 2000: Educate
America Act (the National Skill Standards Act) in assessing studentattainment of occupational competencies at the secondary level, and at the
postsecondary level as appropriate.
(d) INFORMATION PROVIDED BY STATE BOARD
(1) The Committee shall make recommendations to the State Board withrespect to modifying standards and measures to be used under this section,
based on the information provided under paragraph (2) and on the state
needs assessment described under Section 118.
(2) In order to improve the system of measures and standards as directed
under Subsection (a), the State Board shall provide to the Committee
information concerning differing types of standards and measures,including(A) the advantages and disadvantages of each type of standard or
measure;
(B) instances in which such standards and measures have beeneffective;
(C) instances -in -which such standards and measures have not been
effective;
(D) information about standards, measures, and assessmentinstruments already in use statewide that may be incorporated into
the core set of performance measures and standards; and
(E) information provided through federal technic--al assistance as
described under Section 119, as appropriate.
19
28
NCRVE, MDS-821
(3) In the event that the State Board does not accept the Committee'srecommendations made as required by paragraph (1), the State Board shall
set forth in the State plan its reasons for not accepting suchrecommendations.
(e) SUBMISSIONSEach State Board will submit to the Department of Education
as part of its State plan: (1) the measures and standards to be used, and (2) animplementation plan including plans for providing technical assistance to local
recipients of Perkins funds, as required under Section 119.
SEC. 118. PROGRAM EVALUATION AND IMPROVEMENT
(i.) ANNUAL EVALUATIONEach recipient of financial assistance under Part C
of Title H shall annually evaluate the effectiveness of the programs conducted
with assistance under this Act, including programs funded under Title 111 as
determined by the State Board in Section 117(a). This evaluation shall heaccomplished in consultation with teachers, parents, representatives ofindividuals who are members of special populations, and industry representatives.
As part of such evaluation, each such recipient shall
(1) evaluate the effectiveness of programs with respect to performance oneach measure and standard developed under Section 117;
(2) evaluate the performance of special populations in comparison to non-
special populations on each such measure and standard;
(3) evaluate the contribution of the instructional practices and experiences
identified under Section 116(a)(2) to performance on the measures and
standards;
(4) evaluate the contribution of any projects, services, and activitiesconducted with assistance under this Act to performance on the measures
and standards; and
(5) identify and adoptstrategies.to improve performance on the measures and
standards.
(b) LOCAL PROGRAM IMPROVEMENT PLANBeginning not less than 1year after the implementation of the provisions of Section 117, all recipients of
funds under Part C of Title II will develop a local program improvement plan that
summarizes the self-evaluation conducted under Subsection (a). A recipient that
20 4
NCRVE, MDS-821
does not meet the standards under Section 117 must describe in its local program
improvement plan specific strategies for making substantial progress (as defined
by the State Board under Section 115(c)) toward those standards in the coming
year. Such local program improvement plans shall be updated annually.
(c) STATE AND LOCAL JOINT PLAN If, L year after imp1P-nentation of the
plan described in Subsection (b), and each year thereafter, substantial progress in
meeting the standards and measures developed under Section 117 has not been
made, the State shall work jointly with the recipient, and teachers, parents, and
students concerned, to develop a plan for program improvement. Each such plan
shall contain(1) a description of the technical assistance and program activities the State
will provide to enhance the performance of the recipient;
(2) a reasonable timetable to improve recipient performance under the plan;
(3) a description of vocational education strategies the recipient will adopt to
improve performance on the measures; and
(4) if necessary, a description of strategies the recipient will adopt to improve
supplementary services provided to individuals who are members of
special populations.
(d) FURTHER ACTIONThe State shall, in conjunction with the eligiblerecipient, annually review and revise the joint plan developed under Subsection
(c) in order to improve program performance and will continue to do so each
consecutive year until the recipient sustains, for more than 1 year, substantial
progress toward or meets the standards developed under Section 117.
SECTION 119. TECHNICAL ASSISTANCE
(a) FEDERAL ASSISTANCE The Secretary shall provide technical assistance to
the States, directlyor by grant; contract; or cooperative agreement, with respect
to the development and implementation of systems under Section 115 and
subsequent sections, including assistance on
(1) setting performance standards;
(2) evaluating the reliability, validity, and lack of bias of measures and
standards;
21 BEST COPY AVAILABLE
NCRVE, MDS-821
(3) selecting and developing appropriate assessment tools, including methods
for assessing academic gains at the postsecondary level;
(4) coordinating Perkins measures and standards with other evolving andemerging federal initiatives such as National Industry Skill Standards and
School-to-Work performance measures;
(5) collecting and disseminating information on models for using performance
data for program improvement, particularly at the local level; and
(6) planning for and providing technical assistance to local recipients.
In providing such assistance, the Secretary shall utilize existing resources in other
Federal agencies.
(b) STATE ASSISTANCEEach State Board shall provide technical assistance,directly or by grant, contract, or cooperative agreement, to local recipients with
respect to the implementation of Sections 117 and 118, including
(1) assistance on evaluating the effectiveness of programs based on theperformance measures and standards developed under Section 117;
(2) assistance on identifying and adopting appropriate strategies forimproving performance on such measures and standards;
(3) training local administrators and instructors in the use of performance
data to improve vocational courses and programs based on the assessment
conducted under Section 117(a); and
(4) disseminating to local recipients examples of the effective use ofperformance data to improve vocational courses and programs.
Section 102 allocates an annual amount for the technical assistance activities
id mtified under this subsection from the allotment made to each State from funds
appropriated under Section 3(a).
SEC. 120. EVALUATING STATE AND NATIONAL SYSTEMS
(a) REPORTThe Secretary shall submit a report to the appropriate committees of
the Congress not later than the expiration of the 4-year period beginning on the
date of the enactment of the Carl D. Perkins Vocational and Applied Technology
Education Act Amendments of 1995. Such report shall include-
(1) a detailed description of the status of each State's system for improving
vocational education at the State and local level developed as required by
Section 115 and subsequent sections;
22 31.
NCRVE, MDS-821
(2) an assessment of the reliability, validity, and lack of bias of the measures
and standards adopted under Section 117; and
(3) an assessment of the degree to which the use of these systems have led to
program improvement at the State and local levels.
23 32
NCRVE, MDS-821
REFERENCES
Sommerfeld, M. (1994, December 7). Voc.-Ed. law expected to get close scrutiny.
Education Week, pp. 18, 21.
Stecher, B. M., & Hanser, L. M. (1995, May). Accountability and Workforce Training
(IP-150). Santa Monica, CA: RAND.
Stecher, B. M., Rahn, ivi. L., Hamer, L. M., Levesque, K., Hoachiander, E. G., Emanuel,
D., Hallmark, B., & Klein, S. G. (1994). Improving Perkins II performancemeasures and standards: Lessons learned from early implementers in four states
(MR-526) (MDS-732). Santa Monica and Berkeley: RAND and the National
Center for Research in Vocational Education, University of California atBerkeley.
NB
1110
111
11N
S U
M11
11 IN
Sal
1111
1N
M S
U 1
011
ES
OM
MI V
IIIIN
N
TE
CH
NIC
AL
APP
EN
DIX
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 115
. IM
PRO
VIN
G V
OC
AT
ION
AL
Sect
ion
115(
a) o
f th
e 19
95 P
erki
ns A
ct is
a n
ewE
DU
CA
TIO
N A
T T
HE
ST
AT
E A
ND
LO
CA
Lse
ctio
n, th
e pu
rpos
e of
whi
ch is
to s
how
how
LE
VE
Lsu
bseq
uent
Sec
tions
116
to 1
19 c
ombi
ne to
(a)
GE
NE
RA
L A
UT
HO
RIT
YE
ach
Sta
tefo
rm a
com
preh
ensi
ve s
yste
m f
or im
prov
ing
Boa
rd r
ecei
ving
fund
s un
der
this
Act
sha
llvo
catio
nal e
duca
tion
at th
e lo
cal a
ndst
ate
deve
lop
and
impl
emen
t a s
yste
m fo
rle
vels
.M
ost o
f th
e el
emen
ts d
escr
ibed
und
erim
prov
ing
voca
tiona
l edu
catio
n at
the
Sta
tene
w S
ectio
n 11
5(a)
wer
e in
clud
ed in
the
olde
ran
d lo
cal l
evel
that
incl
udes
:le
gisl
atio
n, b
ut s
tate
s w
ere
not d
irec
ted
in th
e(1
) re
appo
intin
g a
Sta
te C
omm
ittee
of
olde
r le
gisl
atio
n to
coo
rdin
ate
thes
e ac
tiviti
esP
ract
ition
ers
(Sec
tion
115)
;no
r to
env
isio
n th
em a
s pa
rt o
f an
ove
rall
(2)c
ondu
ctin
g a
Sta
te n
eeds
ass
essm
ent i
nth
e fir
st y
ear
unde
r re
auth
oriz
atio
n an
dev
ery
two
year
s th
erea
fter
(Sec
tion
116)
;
impr
ovem
ent s
yste
m.
New
Sec
tion
115(
a) in
clud
es a
s co
mpo
nent
s of
(3)i
mpl
emen
ting
a se
t of c
ore
perf
orm
ance
an o
vera
ll im
prov
emen
t sys
tem
the
Com
mitt
eem
easu
res
and
stan
dard
s (S
ectio
n 11
7);
of P
ract
ition
ers,
Sta
te n
eeds
ass
essm
ent,
core
(4)
cond
uctin
g an
nual
eva
luat
ions
of
perf
orm
ance
mea
sure
s an
d st
anda
rds,
and
prog
ram
effe
ctiv
enes
s (S
ectio
n 11
8);
and
annu
al p
rogr
am e
valu
atio
ns, a
ll of
whi
ch w
ere
(5)
prov
idin
g te
chni
cal a
ssis
tanc
e to
loca
lin
trod
uced
in th
e 19
90 A
ct. A
dditi
onal
ly, t
here
cipi
ents
on
usin
g pe
rfor
man
ce d
ata
to19
95 A
ct a
dds
an e
mph
asis
on
prov
idin
gim
prov
e vo
catio
nal c
ours
es a
ndte
chni
cal a
ssis
tanc
e to
loca
l rec
ipie
nts
as a
key
prog
ram
s (S
ectio
n 11
9).
com
pone
nt o
f th
e im
prov
emen
t sys
tem
.T
his
syst
em s
hall
build
on
the
syst
emin
itiat
ed u
nder
the
1990
Per
kins
Act
, and
The
con
cept
s th
at w
ere
cove
red
in S
ectio
n 11
5ap
ply
to a
ll pr
ogra
ms
(as
defin
ed in
of th
e 19
90 P
erki
ns A
ctth
e C
omm
ittee
of
Sub
sect
ion
(c))
rec
eivi
ng fu
nds
unde
r T
itle
Prac
titio
ners
and
the
core
per
form
ance
II, a
nd a
s ap
prop
riate
, to
prog
ram
s fu
nded
mea
sure
s an
d st
anda
rdsa
re in
clud
ed in
the
unde
r T
itle
III
of th
is A
ct.
prop
osed
rev
ised
legi
slat
ion
unde
r Se
ctio
ns11
5(b)
and
117
.
00
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 115
. (b)
RE
QU
IRE
ME
NT
Eac
h St
ate
SEC
. 115
. (a)
GE
NE
RA
L A
UT
HO
RIT
YT
he 1
995
Act
add
s to
the
list o
f co
nstit
uenc
ies
Boa
rd, b
efor
e th
e ex
pira
tion
of th
e 30
-day
Eac
h St
ate
Boa
rd r
ecei
ving
fun
ds u
nder
this
with
whi
ch th
e St
ate
Boa
rd m
ust c
onsu
lt in
peri
od b
egin
ning
on
the
date
of
the
enac
tmen
t of
Act
, bef
ore
the
expi
ratio
n of
the
30-d
ay p
erio
dap
poin
ting
th4
Com
mitt
ee o
f Pr
actit
ione
rs: (
1)th
e19
95 P
erki
ns A
ct, s
hall
reap
poin
t aSt
ate
begi
nnin
g on
the
date
of
the
enac
tmen
t of
the
the
Stat
e ad
min
istr
ator
of
prog
ram
s de
velo
ped
Com
mitt
ee o
f Pr
actit
ione
rs (
refe
rred
to a
s th
eC
arl D
. Per
kins
Voc
atio
nal a
nd A
pplie
dun
der
Titl
e II
, Sub
title
A o
f th
e Sc
hool
-to-
Wor
k"C
omm
ittee
")to
dev
elop
the
elem
ents
ofa
Tec
hnol
ogy
Edu
catio
n A
ct A
men
dmen
ts o
fO
ppor
tuni
ties
Act
of
1994
, and
(2)
the
Stat
epr
ogra
m im
prov
emen
t sys
tem
con
tain
ed u
nder
1990
, sha
ll ap
poin
t the
Sta
te C
omm
ittee
of
adm
inis
trat
or o
f pr
ogra
ms
assi
sted
und
erT
itle
the
abov
e ge
nera
l aut
horit
y. C
omm
ittee
Prac
titio
ners
(in
this
sec
tion
refe
rred
to a
s th
eII
I of
the
Goa
ls 2
000:
Edu
cate
Am
eric
a A
ct.
mem
bers
will
be
appo
inte
d in
con
sulta
tion
with
"Com
mitt
ee")
as p
resc
ribe
d by
Sec
tion
512(
a)C
onsu
lting
with
thes
e ad
min
istr
ator
s w
ill a
ssis
tlo
cal s
choo
l off
icia
ls r
epre
sent
ing
elig
ible
afte
r co
nsul
ting
with
loca
l sch
ool o
ffic
ials
in c
oord
inat
ing
prog
ram
impr
ovem
ent a
ndre
cipi
ents
, and
rep
rese
ntat
ives
of
orga
nize
dre
pres
entin
g el
igib
le r
ecip
ient
s, a
ndpe
rfor
man
ce m
easu
rem
ent s
trat
egie
s am
ong
the
labo
r, b
usin
ess,
sup
erin
tend
ents
, com
mun
ity-
base
d or
gani
zatio
ns, p
riva
te in
dust
ry c
ounc
ilsre
pres
enta
tives
of
orga
nize
d la
bor,
bus
ines
s,su
peri
nten
dent
s, c
omm
unity
-bas
ed19
95 P
erki
ns A
ct, S
choo
l-to
-Wor
kO
ppor
tuni
ties
Act
, and
Goa
ls 2
000
initi
ativ
es in
esta
blis
hed
unde
r Se
ctio
n 10
2(a)
of
the
Job
orga
niza
tions
, pri
vate
indu
stry
cou
ncils
each
sta
te. T
his
coor
dina
tion
dire
ctiv
e is
Tra
inin
g Pa
rtne
rshi
p A
ct, S
tate
cou
ncils
,pa
rent
s, s
peci
al p
opul
atio
ns, c
orre
ctio
nal
inst
itutio
ns, t
he a
dmin
istr
ator
app
oint
ed u
nder
esta
blis
hed
unde
r Se
ctio
n 10
2(a)
of
the
Job
Tra
inin
g Pa
rtne
rshi
p A
ct, S
tate
cou
ncils
,pa
rent
s, s
peci
al p
opul
atio
ns, c
orre
ctio
nal
rein
forc
ed in
new
Sec
tion
117(
c).
Sect
ion
111(
b)(1
), th
e St
ate
adm
inis
trat
or o
fin
stitu
tions
, the
adm
inis
trat
or a
ppoi
nted
und
erpr
ogra
ms
assi
sted
und
er P
art B
of
the
Edu
catio
nSe
ctio
n 11
1(b)
(1),
the
Stat
e ad
min
istr
ator
of
of th
e H
andi
capp
ed A
ct, t
he S
tate
adm
inis
trat
orpr
ogra
ms
assi
sted
und
er P
art B
of
the
of p
rogr
ams
assi
sted
und
er C
hapt
er 1
of
Titl
e I
Edu
catio
n of
the
Han
dica
pped
Act
, the
Sta
teof
the
Ele
men
tary
and
Sec
onda
ry E
duca
tion
adm
inis
trat
or o
f pr
ogra
ins
assi
sted
und
erA
ct, t
he S
tate
adm
inis
trat
or o
f pr
ogra
ms
for
Cha
pter
1 o
f T
itle
I of
the
Ele
men
tary
and
stud
ents
of
limite
d E
nglis
h pr
ofic
ienc
y, a
ndSe
cond
ary
Edu
catio
n A
ct, t
he S
tate
guid
ance
cou
nsel
ors,
and
addi
tiona
llyth
ead
min
istr
ator
of
prog
ram
s fo
r st
uden
ts o
fS
tate
adm
inis
trat
or o
f pro
gram
s de
velo
ped
limite
d E
nglis
h pr
ofic
ienc
y, a
nd g
uida
nce
unde
r T
itle
II, S
ubtit
le A
of t
he S
choo
l-to-
Wor
kco
unse
lors
.O
ppor
tuni
ties
Act
of 1
994,
and
the
Sta
tead
min
istr
ator
of p
rogr
ams
assi
sted
und
er T
itle
111
of th
e G
oals
200
0: E
duca
te A
mer
ica
Act
..
5 -1
Az
rn
IIIIII
IleE
ll O
MIM
O11
1111
IIIIII
OM
VIM
UM
MD
MI B
IMN
IB S
I
1111
111
INS
1111
1M
B 1
1111
illM
il M
I MI M
INR
IM O
N IM
O IN
1111
11IO
N11
1111
1=
IP
14
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SE
C.
115.
(c)
DE
FIN
ITIO
NS
The
Sta
teN
ew S
ectio
n 11
5(c)
cla
rifi
es th
e de
fini
tions
of
Boa
rdsh
all a
dopt
the
follo
win
g de
finiti
ons
in it
sou
tcom
e, m
easu
re, a
nd s
tand
ard;
pro
vide
s a
Sta
te p
lan:
defi
nitio
n of
voc
atio
nal p
rogr
ams;
rev
ises
the
( I)
Ow
com
ea m
easu
rabl
e ch
arac
teris
tic o
fde
fini
tion
of s
peci
al p
opul
atio
ns, r
epla
cing
stud
ent o
r pr
ogra
m p
erfo
rman
ce;
"ind
ivid
uals
who
par
ticip
ate
in p
rogr
ams
(2)
Mea
sure
a m
etho
d fo
r qu
antif
ying
desi
gned
to e
limin
ate
sex
bias
" w
ithpe
rfor
man
ce in
an
outc
ome
area
;"i
ndiv
idua
ls in
pro
gram
s no
ntra
ditio
nal f
or th
eir
(3)
Sta
ndar
dthe
des
ired
leve
l of a
ttain
men
t on
sex"
; and
req
uire
s th
e St
ate
Boa
rd to
def
ine
the
a m
easu
re, o
r th
e ra
te o
f pro
gres
s on
ate
rm"s
ubst
antia
l pro
gres
s" f
or u
se b
y lo
cal
mea
sure
;re
cipi
ents
in th
eir
annu
al p
rogr
am e
valu
atio
ns.
(4)
Pro
gram
a se
quen
ce o
f cou
rses
or
inst
ruct
ion
in a
seq
uenc
e or
agg
rega
tion
ofoc
cupa
tiona
l com
pete
naie
s th
at in
clud
esoc
cupa
tiona
lly s
peci
fic s
kills
, gen
eral
empl
oyab
ility
ski
lls, w
ork
attit
udes
, app
lied
acad
emic
lear
ning
, hig
her-
orde
r re
ason
ing,
and
prob
lem
-sol
ving
ski
lls;
(5)
Spe
cial
pop
ulat
ions
incl
udes
indi
vidu
als
with
dis
abili
ties,
edu
catio
nally
and
econ
omic
ally
dis
adva
ntag
ed in
divi
dual
s(in
clud
ing
fost
er c
hild
ren)
, ind
ivid
uals
of
limite
d E
nglis
h pr
ofic
ienc
y, in
divi
dual
s in
prog
ram
s no
ntra
ditio
nal f
or th
eir
sex,
and
indi
vidu
als
in c
orre
ctio
nal i
nstit
utio
ns.
The
Sta
te B
oard
sha
ll al
so d
efin
e th
e te
rm"s
ubst
antia
l pro
gres
s" in
its
Sta
te p
lan
so th
atth
e te
rm is
mea
sura
ble
and
appl
ies
cons
iste
ntly
to a
ll re
cipi
ents
of P
erki
ns fu
nds
in th
e st
ate.
3 8
00
tr) O
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 116
. ST
AT
E N
EE
DS
ASS
ESS
ME
NT
(a)
IN G
EN
ER
AL
Eac
h St
ate
Boa
rd r
ecei
ving
assi
stan
ce u
nder
this
Act
sha
ll co
nduc
t an
asse
ssm
ent u
sing
mea
sura
ble
obje
ctiv
e cr
iteri
ade
velo
ped
by th
e St
ate
Boa
rd to
ass
ess
prog
ram
qual
ity. S
uch
crite
ria
shal
l. be
dev
elop
ed in
cons
ulta
tion
with
rep
rese
ntat
ives
of
the
Com
mitt
ee d
escr
ibed
in S
ectio
n 11
5(b)
and
sha
llus
e in
form
atio
n ga
ther
ed b
y th
e N
atio
nal
Occ
upat
iona
l Inf
orm
atio
n C
oord
inat
ing
Com
mitt
ee a
nd, a
s ap
prop
riat
e, o
ther
info
rmat
ion.
Eac
h St
ate
Boa
rd s
hall
wid
ely
diss
emin
ate
such
cri
teri
a. S
tate
Boa
rds
shal
lde
velo
p su
ch c
rite
ria
no la
ter
than
the
begi
nnin
gof
the
1996
-199
7 sc
hool
yea
r. S
uch
crite
ria
shal
lin
clud
e(1
) In
form
atio
n on
out
com
es(A
) st
uden
t gai
ns in
and
/or
atta
inm
ent o
fac
adem
ic, o
ccup
atio
nal,
and/
or g
ener
ic w
ork
skill
s;(B
) st
uden
t ret
entio
n in
and
/or
com
plet
ion
ofse
cond
ary
or p
osts
econ
dary
edu
catio
n an
d/or
voca
tiona
l pro
gram
s;(C
) st
uden
t pla
cem
ent i
nto
addi
tiona
l tra
inin
g or
educ
atio
n, m
ilita
ry s
ervi
ce, a
nd e
mpl
oym
ent;
(D)
othe
r ou
tcom
e ar
eas
iden
tifie
d by
the
Stat
eun
der
Sect
ion
115
of th
e 19
90 P
erki
ns A
ct;
(2)
Info
rmat
ion
on in
stru
ctio
nal p
ract
ices
and
expe
rien
ces
(A)
inte
grat
ion
of a
cade
mic
and
voc
atio
nal
educ
atio
n;(B
) th
e ab
ility
of
the
elig
ible
rec
ipie
nts
to m
eet
the
need
s of
spe
cial
pop
ulat
ions
with
res
pect
to v
ocat
iona
l edu
catio
n, in
clud
ing
prov
idin
gin
cent
ives
for
par
ticip
atio
n, r
eten
tion,
and
com
plet
ion;
SEC
. 116
. ST
AT
E A
SrE
SSM
EN
T(a
) IN
G'E
NE
RA
LE
ach
Stat
e B
oard
rece
ivin
g as
sist
ance
und
er th
is A
ct s
hall
cond
uct a
n as
sess
men
t usi
ng m
easu
rabl
eob
ject
ive
crite
ria
deve
lope
d by
the
Stat
eB
oard
to a
sses
s pr
ogra
m q
ualit
y. S
uch
crite
ria
shal
l be
deve
lope
d in
con
sulta
tion
with
rep
rese
ntat
ives
of
the
grou
ps d
escr
ibed
in S
ectio
n 11
5(a)
and
sha
ll us
e in
form
atio
nga
ther
ed b
y th
e N
atio
nal O
ccup
atio
nal
Info
rmat
ion
Coo
rdin
atin
g C
omm
ittee
and
,if
app
ropr
iate
, oth
er in
form
atio
n. E
ach
Stat
e B
oard
sha
ll w
idel
y di
ssem
inat
e su
chcr
iteri
a. S
tate
Boa
rds
shal
l dev
elop
suc
hcr
iteri
a no
late
r th
an th
e be
ginn
ing
of th
e19
91-1
992
scho
ol y
ear.
Suc
h cr
iteri
a sh
all
incl
ude
such
fac
tors
as
(3)
incr
ease
d st
uden
t wor
k sk
ill a
ttain
men
tan
d jo
b pl
acer
icie
nt;
(1)
inte
grat
ion
of a
cade
mic
and
voc
atio
nal
educ
atio
n;(6
) th
e ab
ility
of
the
elig
ible
rec
ipie
nts
tom
eet t
he n
eeds
of
spec
ial p
opul
atio
nsw
ith r
espe
ct to
voc
atio
nal e
duca
tion;
All
of th
e cr
iteri
a re
quir
ed b
y Se
ctio
n 11
6(a)
of
the
1990
Per
kins
Act
to b
e in
clud
ed in
the
Stat
ene
eds
asse
ssm
ent i
s co
ntai
ned
in n
ew S
ectio
n11
6(a)
. How
ever
, new
Sec
tion
116(
a) a
lso
adds
seve
ral c
rite
ria
and
regr
oups
the
crite
ria
into
thre
e gr
oups
: (1)
info
rmat
ion
on o
utco
mes
(mos
tly n
ew),
(2)
info
rmat
ion
on in
stru
ctio
nal
prac
tices
and
exp
erie
nces
, and
(3)
pro
cedu
res
for
usin
g ex
istin
g re
sour
ces
and
met
hods
deve
lope
d in
oth
er p
rogr
ams
rece
ivin
g fe
dera
las
sist
ance
.
In p
artic
ular
, new
Sec
tion
116(
a)(1
) di
rect
s th
eSt
ate
Boa
rd t
exam
ine
outc
ome
data
gen
erat
edth
roug
h th
per
form
ance
mea
sure
s an
dst
anda
rds
syst
em d
evel
oped
und
er th
e 19
90Pe
rkin
s A
ct .n
con
duct
ing
its n
eeds
ass
essm
ent.
Sect
ion
I 16
(a)(
2) r
equi
res
the
Stat
e B
oard
toex
amin
e th
e in
stru
ctio
nal p
ract
ices
and
expe
rien
ces
that
con
trib
ute
to c
reat
ing
qual
ityvo
catio
nal e
duca
tion
prog
ram
s. S
ubse
quen
tly,
in n
ew S
ectio
n 11
8(a)
(3),
loca
l rec
ipie
nts
are
requ
ired
to c
onsi
der
the
cont
ribu
tion
of th
ese
inst
ruct
iona
l pra
ctic
es a
nd e
xper
ienc
es to
thei
rpe
rfor
man
ce o
n th
e co
re m
easu
res
and
stan
dard
s.
A (f)
t..)
41
1111
1111
1al
l --
IIIIM
O In
III S
S N
MI
MIN
1
IIII
III
INS
UM
MI
EM
III
III
III
all M
III
III
MIN
IIII
IIII
ON
MI
8111
11M
I G
NI
1111
11
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 116
. ST
AT
EN
EE
DS
ASS
ESS
ME
NT
cont
inue
d(C
) as
sess
ing
the
qual
ity o
f vo
catio
nal
educ
atio
n pr
ogra
ms
in s
choo
ls w
ith h
igh
conc
entr
atio
ns o
f po
oror
low
-ach
ievi
ngst
uden
ts;
(D)
enro
llmen
t, re
tent
ion,
and
com
plet
ion
rate
s of
stu
dent
s in
voc
atio
nal p
rogr
ams
nont
radi
tiona
l for
thei
r se
x;(E
) se
quen
tial c
ours
es o
f st
udy
lead
ing
tobo
th a
cade
mic
and
occ
upat
iona
lco
mpe
tenc
ies;
(F)
linka
ges
betw
een
seco
ndar
y an
dpo
stse
cond
ary
educ
atio
nal i
nstit
utio
ns;
(G)
inst
ruct
ion
and
expe
rien
ce, t
o th
e ex
tent
prac
ticab
le, i
n al
l asp
ects
of
the
indu
stry
the
stud
ents
are
pre
pari
ng to
ent
er;
(H)
the
rele
vanc
e of
pro
gram
s to
the
wor
kpla
ce a
nd to
the
occu
patio
ns f
orw
hich
stu
dent
s ar
e to
be
trai
ned,
and
the
exte
nt to
whi
ch s
uch
prog
ram
s re
flec
t are
alis
ticas
sess
men
t of
curr
ent a
nd f
utur
ela
bor
mar
ket n
eeds
, inc
ludi
ng n
eeds
inar
eas
of e
mer
ging
tech
nolo
gies
;(I
) th
e ab
ility
of
the
voca
tiona
l cur
ricu
lum
,eq
uipm
ent,
and
inst
ruct
iona
lm
ater
ials
tom
eet t
he d
eman
ds o
f th
e w
orkp
lace
;(J
) ba
sic
and
high
er o
rder
cur
rent
and
fut
ure
wor
kpla
ce c
ompe
tenc
ies
that
will
ref
lect
the
hiri
ng n
eeds
of
empl
oyer
s;(K
) ot
her
fact
ors
cons
ider
ed a
ppro
pria
te b
yth
e St
ate
Boa
rd; a
nd(3
)P
roce
dure
s fo
r us
ing
exis
ting
reso
urce
s an
dm
etho
ds d
evel
oped
in o
ther
pro
gram
sre
ceiv
ing
Fed
eral
ass
ista
nce.
4'2
SEC
. 116
. ST
AT
E A
SSE
SSM
EN
Tco
ntin
ued
(7)
rais
ing
the
qual
ity o
f vo
catio
nal e
duca
tion
prog
ram
s in
sch
ools
with
hig
hco
ncen
trat
ions
of
poor
and
low
-ach
ievi
ngst
uden
ts;
(2)
sequ
entia
l cou
rse
of s
tudy
lead
ing
to b
oth
acad
emic
and
occ
upat
iona
l com
pete
ncie
s;
(4)
incr
ease
! lin
kage
s be
twee
n se
cond
ary
and
post
seco
ndar
y ed
ucat
iona
l ins
titut
ions
;(5
) in
stru
ctio
n an
d ex
peri
ence
, to
the
exte
ntpr
actic
able
, in
all a
spec
ts o
f th
e in
dust
ry th
est
uden
ts a
re p
repa
ring
to e
nter
;(8
) th
e re
leva
nce
of p
rogr
ams
to th
e w
orkp
lace
and
to th
e oc
cupa
tions
for
whi
ch s
tude
nts
are
to b
e tr
aine
d, a
nd th
e ex
tent
to w
hich
such
pro
gram
s re
flec
t a r
ealis
tic a
sses
smen
tof
cur
rent
and
fut
ure
labo
r m
arke
t nee
ds,
incl
udin
g ne
eds
in a
reas
of
emer
ging
tech
nolo
gies
;(9
) th
e ab
ility
of
the
voca
tiona
l cur
ricu
lum
,eq
uipm
ent,
and
inst
ruct
iona
l mat
eria
ls to
mee
t the
dem
ands
of
the
wor
kpla
ce;
(10)
bas
ic a
nd h
ighe
r or
der
curr
ent a
nd f
utur
ew
orkp
lace
com
pete
ncie
s w
hich
will
ref
lect
the
hiri
ng n
eeds
of
empl
oyer
s; a
nd(1
1) o
ther
fac
tors
con
side
red
appr
opri
ate
by th
eSt
ate
Boa
rd.
New
Sec
tion
116(
a)(2
)(D
) re
flec
ts th
e se
xeq
uity
con
cern
s in
the
1990
Per
kins
Act
. The
seco
ncer
ns a
re a
gain
ech
oed
in n
ew S
ectio
n11
7(b)
(5)
as p
art o
f th
e co
re p
erfo
rman
cem
easu
res
and
stan
dard
s.
New
Sec
tion
116(
a)(3
) w
as c
onta
ined
in S
ectio
n11
5(b)
(4)
of th
e 19
90 P
erki
ns A
ct a
s pa
rt o
f th
esy
stem
of
perf
orm
ance
mea
sure
s an
d st
anda
rds.
We
belie
ve it
is m
ore
appr
opri
ate
to e
xam
ine
thes
e pr
oced
ures
as
pan
of th
e St
ate
need
sas
sess
men
t.
43
A 00 ts.)
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 116
. (b)
DE
AD
LIN
E F
OR
ASS
ESS
ME
NT
Eac
h St
ate
Boa
rd s
hall
com
plet
e th
e as
sess
men
t req
uire
d by
Sub
sect
ion
(a)
befo
re th
e ex
pira
tion
of th
e 6-
mon
th p
erio
dbe
ginn
ing
on th
e da
te o
f th
e en
actm
ent o
f th
e19
95 P
erki
ns A
ct, a
nd e
very
two
year
sth
erea
fter
.
SEC
. 117
. ST
AT
E A
ND
LO
CA
LM
EA
SUR
ES
AN
D S
TA
ND
AR
DS
(a)
GE
NE
RA
L A
UT
HO
RIT
YE
ach
Stat
eB
oard
rec
eivi
ng f
unds
und
er th
is A
ct s
hall
revi
se a
nd im
plem
ent a
set
of
core
sta
ndar
dsan
d m
easu
res
of p
erfo
rman
ce f
or s
econ
dary
and
post
seco
ndar
y vo
catio
nal e
duca
tion
prog
ram
s. T
o ad
vise
the
Stat
e B
oard
, the
Com
mitt
ee s
hall
(1)
eval
uate
the
relia
bilit
y, v
alid
ity, a
nd la
ckof
bia
s of
the
mea
sure
s an
d st
anda
rds
deve
lope
d un
der
Sect
ion
115
of th
e 19
90Pe
rkin
s A
ct;
(2)
asse
ss th
e ex
tent
to w
hich
dat
a ge
nera
ted
unde
r Se
ctio
n 11
5 of
the
1990
Per
kins
Act
wer
e us
ed e
ffec
tivel
y by
rec
ipie
nts
toim
prov
e pr
ogra
ms
and
iden
tify
barr
iers
to e
ffec
tive
use:
(3)
reco
mm
end
revi
sion
s to
exi
stin
gm
easu
res
and
stan
dard
s; a
nd(4
) de
velo
p ne
w m
easu
res
and
stan
dard
s as
requ
ired
und
er S
ubse
ctio
n (b
).
SEC
. 116
. (b)
DE
AD
LIN
E F
OR
ASS
ESS
ME
NT
Eac
h St
ate
Boa
rd s
hall
com
plet
e th
e as
sess
men
t req
uire
d by
Subs
ectio
n (a
) be
fore
the
expi
ratio
n of
the
6-m
onth
per
iod
begi
nnin
g on
the
date
of
the
enac
tmen
t of
the
Car
l D. P
erki
ns V
ocat
iona
lan
d A
pplie
d T
echn
olog
y E
duca
tion
Act
Am
endm
ents
of
1990
.
SEC
. 115
. (a)
GE
NE
RA
L A
UT
HO
RIT
YE
ach
Stat
e B
oard
rec
eivi
ng f
unds
und
er th
isA
ct s
hall
deve
lop
and
impl
emen
t a s
tate
wid
esy
stem
of
core
sta
ndar
ds a
nd m
easu
res
ofpe
rfor
man
ce f
or s
econ
dary
and
pos
t-se
cond
ary
voca
tiona
l edu
catio
n pr
ogra
ms.
44
New
Sec
tion
116(
b) r
equi
res
the
Stat
e B
oard
toco
nduc
t a n
eeds
ass
essm
ent e
very
two
year
s,m
akin
g th
e ne
eds
asse
ssm
ent a
reg
ular
act
ivity
as p
art o
f th
e sy
stem
for
impr
ovin
g vo
catio
nal
educ
atio
n.
New
Sec
tion
117(
a) d
irec
ts th
e St
ate
Boa
rd to
revi
se th
e co
re m
easu
res
and
stan
dard
sde
velo
ped
unde
r th
e 19
90 P
erki
ns A
ct w
ithre
gard
to te
chni
cal q
ualit
y an
d th
e ef
fect
iven
ess
of th
eir
past
use
for
impr
ovin
g pr
ogra
ms.
The
new
sec
tion
also
elim
inat
es u
se o
f th
e te
rm"s
yste
m"
whe
n re
ferr
ing
to th
e co
re m
easu
res
and
stan
dard
s, r
eser
ving
that
term
inst
ead
for
the
over
all "
syst
em"
for
impr
ovin
g vo
catio
nal
educ
atio
n as
des
crib
ed in
new
Sec
tion
115(
a).
45
MI M
I MI N
M N
MI
1111
111
1111
11M
EI N
I NM
NS
GM
GN
IO
NIII
II11
11
Ma
1111
1In
an
1111
1111
11In
IM
P11
11 N
M 1
111
1111
GM
UM
NM
NM
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 117
. (a)
cont
inue
dSE
C. 1
15. (
a) c
ontin
ued
Und
er n
ew S
ectio
n 11
7(a)
the
Stat
e B
oard
isR
evis
ions
to th
e co
re m
easu
res
and
stan
dard
s.
..su
ch s
yste
m s
hall
be d
evel
oped
and
enco
urag
ed to
ext
end
the
core
per
form
ance
will
be
reco
mm
ende
d by
the
Com
mitt
ee a
ndim
plem
ente
d be
fore
the
end
of th
e 2-
year
mea
sure
s an
d st
anda
rds
to c
over
Titl
e II
Iap
prov
ed b
y ea
ch S
tate
Boa
rd n
o la
ter
han
one
peri
od b
egin
ning
on
the
date
of
the
enac
tmen
tpr
ogra
ms
such
as
cons
umer
and
hom
emak
ing
year
afte
r th
e en
actm
ent o
f the
199
5 P
erki
nsof
the
Car
l D. P
erki
ns V
ocat
iona
l and
App
lied
educ
atio
n an
d T
ech
Prep
.A
ct. T
his
core
sha
ll ap
ply
to a
ll pr
ogra
ms
(as
Tec
hnol
ogy
Edu
catio
n A
ct A
men
dmen
ts o
fde
fined
in S
ectio
n 11
5(cX
4))
rece
ivin
g flu
ids
1990
and
sha
ll ap
ply
to a
ll pr
ogra
ms
assi
sted
The
opt
ion
to m
ake
loca
l mod
ific
atio
ns to
the
unde
r T
itle
II of
this
Act
. Add
ition
ally
, the
Sta
teun
der
this
Act
. Elig
ible
rec
ipie
nts
may
mak
esy
stem
;las
bee
n m
aint
aine
d.B
oard
may
req
uire
that
the
core
app
ly to
any
loca
l mod
ific
atio
ns to
suc
h sy
stem
bas
ed o
npr
ogra
ms
fund
ed u
nder
Titl
e III
suc
h as
cons
umer
and
hom
emak
ing
educ
atio
n an
d te
ch-
econ
omic
, geo
grap
hic,
or
dem
ogra
phic
fac
tors
,or
the
char
acte
rist
ics
of th
e po
pula
tion
to b
epr
ep.
Elig
ible
rec
ipie
nts
may
mak
e lo
cal
serv
ed. S
uch
mod
ific
atio
ns s
hall
conf
orm
tom
odif
icat
ions
to s
uch
core
base
d on
eco
nom
ic,
geog
raph
ic, o
r de
mog
raph
ic f
acto
rs, o
r th
eth
e as
sess
men
t cri
teri
a co
ntai
ned
in th
e St
ate
plan
. The
Sta
te B
oard
sha
ll co
nven
e th
ech
arac
teri
stic
s of
the
popu
latio
n to
be
serv
ed.
Such
mod
ific
atio
ns s
hall
conf
orm
to th
eC
omm
ittee
on
a re
gula
r ba
sis
to r
evie
w,
com
men
t on,
and
pro
pose
rev
isio
ns to
a d
raft
asse
ssm
ent c
rite
ria
cont
aine
d in
the
Stat
e pl
an.
Stat
e pr
opos
al, w
hich
the
Stat
e B
oard
sha
llT
he S
tate
Boa
rd s
halt
conv
ene
the
Com
mitt
eede
velo
p, f
ora
syst
em o
f co
re s
tand
ards
and
on a
reg
ular
bas
is to
rev
iew
, com
men
t on,
and
mea
sure
s of
per
form
ance
for
voc
atio
nal
prop
ose
revi
sion
s to
the
Sta
te's
cor
e st
anda
rds
and
mea
sure
s of
per
form
ance
for
voca
tiona
lpr
ogra
ms.
prog
ram
s.
47
A
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 117
. (b)
RE
QU
IRE
ME
NT
SEac
hsy
stem
dev
elop
ed u
nder
Sub
sect
ion
(a)
shal
lin
clud
e at
leas
t fiv
e m
easu
res
and
stan
dard
s(1
) a
mea
sure
of
stud
ent l
earn
ing
gain
s in
the
achi
evem
ent o
f ac
adem
ic s
kills
;(2
) I
or m
ore
of th
e fo
llow
ing
mea
sure
s of
wor
k-re
late
d sk
ills
(A)
occu
patio
nal c
ompe
tenc
y at
tain
men
t in
the
indu
stry
a s
tude
nt is
pre
pari
ng to
ente
r;(B
) at
tain
men
t of
wor
k re
adin
ess
or g
ener
icw
ork
skill
s;(C
) at
tain
men
t of
unde
rsta
ndin
g of
all
aspe
cts
of th
e in
dust
ry a
stu
dent
ispr
epar
ing
to e
nter
;(3
) 1
or b
oth
of th
e fo
llow
ing
mea
sure
s of
rete
ntio
n or
com
plet
ion
(A)
rete
ntio
n in
or
com
plet
ion
of s
econ
dary
or p
osts
econ
dary
edu
catio
n;(B
) re
tent
ion
in o
r co
mpl
etio
e. o
f se
cond
ary
or p
osts
econ
dary
occ
upat
iona
l pro
gram
sor
oth
er p
rogr
ams
fund
ed u
nder
Titl
e 11
1,as
app
ropr
iate
;(4
) a
mea
sure
of
plac
emen
t int
o ad
ditio
nal
trai
ning
or
educ
atio
n (i
nclu
ding
pla
cem
ent
into
the
post
seco
ndar
y co
mpo
nent
of
tech
-pr
ep o
r ot
her
Scho
ol-t
o-W
ork
tran
sitio
npr
ogra
m),
mili
tary
ser
vice
, or
empl
oym
ent;
and
(5)
a m
easu
re o
f ac
cess
and
equ
ity s
uch
as r
ates
of e
nrol
lmen
t in,
ret
entio
n in
, and
/or
com
plet
ion
of o
ccup
atio
nal o
r ot
her
prog
ram
s fo
r sp
ecia
l pop
ulat
ions
inco
mpa
riso
n to
non
-spe
cial
pop
ulat
ions
,in
clud
ing
mal
e an
d fe
mal
e st
uden
ts in
prog
ram
s no
ntra
ditio
nal f
or th
eir
sex.
SEC
. 115
. (b)
RE
QU
IRE
ME
NT
SE
ach
syst
em d
evel
oped
und
er S
ubse
ctio
n (a
)sh
all i
nclu
de(1
) m
easu
res
of le
arni
ng a
nd c
ompe
tenc
yga
ins,
incl
udin
g st
uden
t pro
gres
s in
the
achi
evem
ent o
f ba
sic
and
mor
e ad
vanc
edac
adem
ic s
kills
;(2
) 1
or m
ore
mea
sure
s of
per
form
ance
, whi
chsh
all i
nclu
de o
nly
(A)
com
pete
ncy
atta
inm
ent;
(B)
job
or w
ork
skill
atta
inm
ent o
ren
hanc
emen
t inc
ludi
ng s
tude
nt p
rogr
ess
in a
chie
ving
occ
upat
iona
l ski
llsne
cess
ary
to o
btai
n em
ploy
men
t in
the
fiel
d fo
r w
hich
the
stud
ent h
as b
een
prep
ared
, inc
ludi
ng o
ccup
atio
nal s
kills
in th
e in
dust
ry th
e st
uden
t is
prep
arin
gto
ent
er;
(C)
rete
ntio
n in
sch
ool o
r co
mpl
etio
n of
seco
ndar
y sc
hool
or
its e
quiv
alen
t; an
d(D
) pl
acem
ent i
nto
addi
tiona
l tra
inin
g or
educ
atio
n, m
ilita
ry s
ervi
ce, o
rem
ploy
men
t;(3
) in
cent
ives
or
adju
stm
ents
that
are
(A)
desi
gned
to e
ncou
rage
ser
vice
tota
rget
ed g
roup
s or
spe
cial
pop
ulat
ions
;an
d(B
) fo
r ea
ch s
tude
nt, c
onsi
sten
t with
the
stud
ent's
indi
vidu
aliz
ed e
duca
tion
prog
ram
dev
elop
ed u
nder
Sec
tion
614(
a)(5
) of
the
Edu
catio
n of
the
Han
dica
pped
Act
, whe
re a
ppro
pria
te;
and
(4)
proc
edur
es f
or u
sing
exi
stin
g re
sour
ces
and
met
hods
dev
elop
ed in
oth
er p
rogr
ams
rece
ivin
g Fe
dera
l ass
ista
nce.
4S
New
Sec
tion
117(
b) e
xpan
ds f
rom
two
to f
ive
the
num
ber
of m
easu
res
and
stan
dard
s re
quir
edas
par
t of
the
core
. The
sec
tion
regr
oups
the
mea
sure
men
t cho
ices
und
er th
e ol
d A
ct,
sim
plif
ies
them
, and
add
s a
mea
sure
of
acce
ssan
d eq
uity
.
49
III
Mill
all
MIM
I OM
INS
OM
III
IIIIII
MI
all
ON
O M
I II
I
MI
NM
I N
MN
IBN
MI
1111
iiM
INIM
OII
IIII
1111
1111
IIII
IIN
M I
MO
UP
MI
MI
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 117
. (c)
CO
NSI
STE
NC
Y A
ND
CO
OR
DIN
AT
ION
WIT
H O
TH
ER
PRO
GR
AM
SIn
deve
lopi
ng th
e st
anda
rds
and
mea
sure
s in
clud
ed in
a s
yste
m d
evel
oped
unde
r Su
bsec
tion
(a),
the
Stat
e B
oard
sha
ll(1
) co
nsid
er s
tand
ards
and
mea
sure
s de
velo
ped
unde
r jo
b op
port
uniti
es a
nd b
asic
ski
llstr
aini
ng p
rogr
ams
esta
blis
hed
and
oper
ated
unde
r a
plan
app
rove
d by
the
Secr
etar
y of
Hea
lth a
nd H
uman
Ser
vice
s th
at m
eets
the
requ
irem
ents
of
Sect
ion
402(
0(19
) of
the
Soci
al S
ecur
ity A
ct;
(2)
cons
ider
sta
ndar
ds p
resc
ribe
d by
the
Secr
etar
y of
Lab
or u
nder
Sec
tion
106
of th
eJo
b T
rain
ing
Part
ners
hip
Act
;(3
) co
ordi
nate
with
Sta
te e
ffor
ts to
impl
emen
tpe
rfor
man
ce m
easu
res
deve
lope
d un
der
Titl
e IV
, Sec
tion
402
of th
e Sc
hool
-to-
Wor
kO
ppor
tuni
ties
Act
;(4
) bu
ild o
n th
e co
nten
t and
stu
dent
perf
orm
ance
sta
ndar
ds a
nd r
elat
edas
sess
men
ts d
evel
oped
und
er T
itle
IL P
art B
of th
e G
oals
200
0: E
duca
te A
mer
ica
Act
inas
sess
ing
stud
ent l
earn
ing
gain
s in
the
achi
evem
ent o
f ac
adem
ic s
kills
at t
hese
cond
ary
leve
l; an
d(5
) bu
ild o
n th
e in
dust
ry s
kill
stan
dard
s an
dre
late
d as
sess
men
ts a
nd c
ertif
icat
ions
deve
lope
d un
der
Titl
e V
of
the
Goa
ls 2
000:
Edu
cate
Am
eric
a A
ct (
the
Nat
iona
l Ski
llSt
anda
rds
Act
) in
ass
essi
ng s
tude
ntat
tain
men
t of
occu
patio
nal c
ompe
tenc
ies
atth
e se
cond
ary
leve
l, an
d at
the
post
seco
ndar
y le
vel a
s ap
prop
riat
e.
SEC
. 115
. (c)
CO
NSI
STE
NC
Y W
ITH
OT
HE
R P
RO
GR
AM
SIn
deve
lopi
ng th
est
anda
rds
and
mea
sure
s in
clud
ed in
a s
yste
mde
pelo
ped
unde
r Su
bsec
tion
(a),
the
Stat
eB
oard
sha
ll ta
ke in
to c
onsi
dera
tion
(1)
stan
dard
s an
d m
easu
res
deve
lope
d un
der
job
oppo
rtun
ities
and
bas
ic s
kills
trai
ning
prog
ram
s es
tabl
ishe
d an
d op
erat
ed u
nder
apl
an a
ppro
ved
by th
e Se
cret
ary
of H
ealth
and
Hum
an S
ervi
ces
that
mee
ts th
ere
quir
emen
ts o
f Se
ctio
n 40
2(a)
(19)
of
the
Soci
al S
ecur
ity A
ct; a
nd(2
) st
anda
rds
pres
crib
ed b
y th
e Se
cret
ary
ofL
abor
und
er S
ectio
n 10
6 of
the
Job
Tra
inin
g Pa
rtne
rshi
p A
ct.
In a
dditi
on to
con
side
ring
sta
ndar
ds d
evel
oped
unde
r JO
BS
and
JTPA
, new
Sec
tion
117(
c) a
dds
the
requ
irem
ent t
o co
ordi
nate
with
sta
ndar
dse
tting
eff
orts
und
er th
e Sc
hool
-to-
Wor
kO
ppor
tuni
ties
Act
and
the
Goa
ls 2
000:
Edu
cate
Am
eric
a A
ct, w
hich
incl
udes
the
Nat
iona
l Ski
llSt
anda
rds
Act
. For
exa
mpl
e, a
cade
mic
stan
dard
s de
velo
ped
unde
r G
oals
200
0 co
uld
beus
ed to
mea
sure
"le
arni
ng g
ains
in th
eac
hiev
emen
t of
acad
emic
ski
lls"
(Sec
.I
1 7(
b)(1
)), a
nd in
dust
ry s
kills
sta
ndar
dsde
velo
ped
unde
r th
e N
atio
nal S
kill
Stan
dard
sA
ct c
ould
be
used
to m
easu
re "
wor
k-re
late
dsk
ills"
(Se
c. 1
17(b
)(2)
).
z oo
199
PER
KIN
S A
CT
1990
PaR
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 117
. (d)
IN
FOR
MA
TIO
N P
RO
VID
ED
BY
STA
TE
BO
AR
D(1
) T
he C
omm
ittee
sha
ll n
ake
reco
mm
enda
tions
to th
eSt
ate
Boa
rd w
ith r
espe
c' to
mod
ifyi
ng s
tand
ards
and
mea
sure
s to
be
used
und
er th
is s
ectio
n, b
ased
on
the
info
rmat
ion
prov
ided
und
er p
arag
raph
(2)
and
on th
est
ate
need
s as
sess
men
t des
crib
ed u
nder
Sec
tion
118.
(2)
In o
rder
to im
prov
e th
e sy
stem
of m
easu
res
and
stan
dard
s as
dire
cted
und
er S
ubse
ctio
n (a
), th
eSt
ate
Boa
rd s
hall
prov
ide
to th
e C
omm
ittee
info
rmat
ion
conc
erni
ng d
iffe
ring
type
s of
sta
ndar
ds a
ndm
easu
res,
incl
udin
g(A
) th
e ad
vant
ages
and
dis
adva
ntag
es o
f ea
ch ty
peof
sta
ndar
d or
mea
sure
;(B
) in
stan
ces
in w
hich
suc
h st
anda
rds
and
mea
sure
sha
ve b
een
effe
ctiv
e;(C
) in
stan
ces
in w
hich
suc
h st
anda
rds
and
mea
sure
sha
ve n
ot b
een
effe
ctiv
e;(D
) in
form
atio
n ab
out s
tand
ards
, mea
sure
s, a
ndas
sess
men
t ins
trum
ents
alre
ady
in u
sest
atew
ide
that
may
be
inco
rpor
ated
into
the
core
set
of p
erfo
rman
ce m
easu
res
and
stan
dard
s; a
nd(E
) in
form
atio
n pr
ovid
ed th
roug
h fe
dera
l tec
hnic
alas
sist
ance
as
desc
ribed
und
er S
ectio
n 11
9, a
sap
prop
riate
.(3
) In
the
even
t tha
t the
Sta
te B
oard
doe
s no
t acc
ept t
heC
omm
ittee
's r
ecom
men
datio
ns m
ade
as r
equi
red
bypa
ragr
aph
(1),
the
Stat
e B
oard
sha
ll se
t for
th in
the
Stat
e pl
an it
s re
ason
s fo
r no
t acc
eptin
g su
chre
com
men
datio
ns.
(e)
SU
BM
ISS
ION
S E
ach
Sta
te B
oard
will
sub
mit
toth
e D
epar
tmen
t of E
duca
tion
as p
art o
f its
Sta
teph
ut: (
1) th
e m
easu
res
and
stan
dard
s to
be
used
.an
d (2
) an
impl
emen
tatio
n pl
an in
clud
ing
plan
s fo
rpr
ovid
ing
tech
nica
l ass
ista
nce
to lo
cal r
ecip
ient
s of
Per
kins
fund
s, a
s re
quire
d un
der
Sec
tion
119.
SEC
. 115
. (d)
RE
Q U
IRE
ME
NT
SIN
FOR
MA
TIO
N P
RO
VID
ED
BY
ST
AT
EB
OA
RD
(1)
The
Com
mitt
ee h
all m
ake
reco
mm
enda
tior
to th
e St
ate
Boa
rd w
ithre
spec
t to
mod
ifyi
ng s
tand
ards
and
mea
sure
s to
be
used
und
er th
is s
ectio
n,ba
sed
on th
z: in
form
atio
n pr
ovid
ed u
nder
para
grap
h (2
).(2
) T
o as
sist
the
Com
mitt
ee in
for
mul
atin
gre
com
men
datio
ns u
nder
par
agra
ph (
1), t
heSt
ate
Boa
rd s
hall
prov
ide
to th
e C
omm
ittee
info
rmat
ion
conc
erni
ng d
iffe
ring
type
s of
stan
dard
s an
d m
easu
res,
incl
udin
g(A
) th
e ad
vant
ages
and
dis
adva
ntag
es o
fea
ch ty
pe o
f st
anda
rd o
r m
easu
re;
(B)
inst
ance
s in
whi
ch s
uch
stan
dard
s an
dm
easu
res
have
bee
n ef
fect
ive;
and
(C)
inst
ance
s in
whi
ch s
uch
stan
dard
s an
dm
easu
res
have
not
bee
n ef
fect
ive.
(3)
In th
e ev
ent t
hat t
he S
tate
Boa
rd d
oes
not
acce
pt th
e C
orri
mitt
ee's
rec
omm
enda
tions
mad
e as
req
uire
d by
par
agra
ph (
1), t
heSt
ate
Boa
rd s
hall
set f
orth
in th
e St
ate
plan
its r
easo
ns f
or n
ot a
ccep
ting
such
reco
mm
enda
tions
.
New
Sec
tion
117(
d) is
sim
ilar
to o
ld S
ectio
n11
5(d)
.H
owev
er, n
ew S
ectio
n 11
7(d)
req
uire
sth
eSt
ate
Boa
rd, i
n m
odif
ying
the
core
, to
exam
ine
the
Stat
e ne
eds
asse
ssm
ent,
as w
ell a
sin
form
atio
n on
sta
ndar
ds, m
easu
res,
and
asse
ssm
ent i
nstr
umen
ts a
lrea
dy in
use
sta
tew
ide
and
info
rmat
ion
prov
ided
thro
ugh
fede
ral
tech
nica
l ass
ista
nce.
New
Sec
tion
117(
e) r
equi
res
the
Stat
e B
oard
tosu
bmit
to th
e D
epar
tmen
t of
Edu
catio
n as
par
tof
its
Stat
e pl
an a
des
crip
tion
of it
s co
rem
easu
res
and
stan
dard
s as
wel
l as
a pl
an f
orpr
ovid
ing
tech
nica
l ass
ista
nce
to lo
cal
reci
pien
ts. T
his
prov
isio
n em
phas
izes
the
impo
itanc
c of
tech
nica
l ass
ista
nce
for
ensu
ring
that
per
form
ance
dat
a w
ill b
e us
ed e
ffec
tivel
yby
loca
l rec
ipie
nts
to im
prov
e pr
ogra
ms.
r
00 ts)
MI
INN
Ell
MN
dill
IMP
NIS
OM
MN
6111
1M
ilO
il N
M M
INII
NII
SIB
NIB
1111
1111
111
NM
ON
O M
I11
.11
MN
NM
1111
111
1111
111
1111
NM
RV
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
SA
CT
CO
MM
EN
TA
RY
SEC
. 118
. PR
OG
RA
ME
VA
LUA
TIO
N A
ND
IMP
RO
VE
ME
NT
(a)
AN
NU
AL
EV
ALU
AT
ION
Eac
h re
cipi
ent o
ffin
anci
al a
ssis
tanc
e un
der
Par
t C o
f Titl
e II
shal
lan
nual
ly e
valu
ate
the
effe
ctiv
enes
s of
the
prog
ram
sco
nduc
ted
with
ass
ista
nce
unde
r th
is A
ct, i
nclu
ding
prog
ram
s fu
nded
und
er T
itle
Ill a
s de
term
ined
by
the
Sta
te B
oard
in S
ectio
n 11
7(a)
. Thi
s ev
alua
tion
shal
l be
acco
mpl
ishe
d in
con
sulta
tion
with
teac
hers
, par
ents
, rep
rese
ntat
ives
of i
ndiv
idua
lsw
ho a
re m
embe
rs o
f spe
cial
pop
ula!
ieis
, and
indu
stry
rep
rese
ntat
ives
. As
part
of s
uch
eval
uatio
n, e
ach
such
rec
ipie
nt s
hall
( 1
) ev
alua
te th
e ef
fect
iven
ess
of p
rogr
ams
with
resp
ect t
o pe
rfor
man
ce o
n ea
ch m
easu
re a
nd..a
ular
d de
velo
ped
unde
r S
ectio
n 11
7;(2
) ev
alua
te th
e pe
rfor
man
ce o
f spe
cial
pop
ulat
ions
in c
ompa
rison
to n
on-s
peci
al p
opul
atio
ns o
nea
ch s
uch
mea
sure
and
sta
ndar
d;(3
) ev
alua
te th
e co
ntrib
utio
n of
the
inst
ruct
iona
lpr
actic
es a
nd e
xper
ienc
es id
entif
ied
unde
rS
ectio
n 11
6(aX
2) to
per
form
ance
on
the
mea
sure
s an
d st
anda
rds;
(4)
eval
uate
the
cont
ribut
ion
of a
ny p
roje
cts,
serv
ices
, and
act
iviti
es c
ondu
cted
with
assi
stan
ce u
nder
this
Act
to p
erfo
rman
ce o
n th
em
easu
res
and
stan
dard
s; a
nd(5
) id
entif
y an
d ad
opt s
trat
egie
s to
impr
ove
perf
orm
ance
on
the
mea
sure
s an
d st
anda
rds.
SE
C. 1
17.P
RO
GR
AM
EV
AL
UA
TIO
NA
ND
IMPR
OV
EM
EN
T(a
) A
NN
UA
L E
VA
LUA
TIO
NE
ach
reci
pien
t of f
inan
cial
ass
ista
nce
unde
r P
art
C o
f Titl
e 11
sha
ll an
nual
ly e
valu
ate
the
effe
ctiv
enes
s of
the
prog
ram
con
duct
edw
ith a
ssis
tanc
e un
der
this
Act
bas
ed o
n th
est
anda
rds
and
mea
sure
s (o
r m
odifi
catio
nsth
eret
o) d
evel
oped
as
requ
ired
by S
ectio
n11
5. A
s pa
rt o
f eac
h su
ch e
valu
atio
n, e
ach
such
rec
ipie
nt s
hall
(1)
revi
ew p
rogr
ams,
with
the
full
and
info
rmed
par
ticip
atio
n of
rep
rese
ntat
ives
of in
divi
dual
s w
ho a
re m
embe
rs o
fsp
ecia
l pop
ulat
ions
, to
(A)
iden
tify
and
adop
t str
ateg
ies
toov
erco
me
any
barr
iers
whi
ch a
rere
sulti
ng in
low
er r
ates
of a
cces
s to
voca
tiona
l edu
catio
n pr
ogra
ms
orsu
cces
s in
suc
h pr
ogra
ms
for
indi
vidu
als
who
are
mem
bers
of
spec
ial p
opul
atio
ns; a
nd(B
) ev
alua
te th
e pr
ogre
ss o
f ind
ivid
uals
who
are
mem
bers
of s
peci
alpo
pula
tions
in v
ocat
iona
l edu
catio
npr
ogra
ms
assi
sted
und
er th
is A
ct;
and
(2)
eval
uate
the
prog
ress
of v
ocat
iona
led
ucat
ion
prog
ram
s as
sist
ed u
nder
this
Act
in p
rovi
ding
voc
atio
nal e
duca
tion
stud
ents
with
str
ong
expe
rienc
e in
and
unde
rsta
ndin
g of
all
aspe
cts
of th
ein
dust
ry th
e st
uden
ts a
re p
repa
ring
toen
ter.
New
Sec
tion
118(
a) r
equi
res
loca
l rec
ipie
nts
toev
alua
te a
nnua
lly th
e ef
fect
iven
ess
of th
eir
Per
kins
-ass
iste
d pr
ogra
ms,
incl
udin
g pr
ogra
ms
fund
ed u
nder
Tid
e Ill
if th
e S
tate
Boa
rd s
ode
term
ined
und
er S
ectio
n i;
17(a
). N
ew S
ectio
n11
8(a)
req
uire
s lo
cal r
ecip
ient
s to
eva
luat
e th
eir
prog
ram
s w
ith r
espe
ct to
stu
dent
per
form
ance
on
each
mea
sure
, as
wel
l as
to d
isag
greg
ate
perf
orm
ance
on
each
mea
sure
for
spec
ial
popu
latio
ns, i
nclu
ding
mal
es a
nd fe
mal
es in
prog
ram
s no
ntra
ditio
nal f
or th
eir
sex.
The
ann
ual
eval
uatio
n m
ust a
lso
cons
ider
the
cont
ribut
ions
of
inst
ruct
iona
l pra
ctic
es a
nd e
xper
ienc
es id
entif
ied
unde
r th
e S
tate
nee
ds a
sses
smen
t and
of P
erki
ns-
w,s
iste
d se
rvic
es, p
roje
cts,
and
act
iviti
es to
perf
orm
ance
on
the
mea
sure
s. T
he c
once
rns
unde
r ol
d S
ectio
ns 1
17(a
)(1)
and
(2)
are
addr
esse
d in
mul
tiple
way
s du
ring
the
annu
alev
alua
tion.
For
inst
ance
, by
requ
iring
loca
lre
cipi
ents
to c
onsi
der
the
cont
ribut
ion
of S
tate
-id
entif
ied
inst
ruct
iona
l pra
ctic
es a
nd e
xper
ienc
esto
per
form
ance
on
the
core
mea
sure
s, n
ewS
ectio
n 11
8(a)
effe
ctiv
ely
requ
ires
reci
pien
ts to
exam
ine
annu
ally
(1)
thei
r ab
ility
to m
eet t
hene
eds
of s
peci
al p
opul
atio
ns (
Sec
tion
116(
a)(2
)(B
)); (
2) th
e pr
eval
ence
of i
nstr
uctio
n in
all a
spec
ts o
f the
indu
stry
(S
ectio
n 11
6(a)
(2)(
G))
,bo
th e
mph
asiz
ed in
Sec
tion
117(
a) o
f the
199
0P
erki
ns A
ct; a
nd (
3) o
ther
inst
ruct
iona
l pra
ctic
es,
such
as
inte
grat
ion
of v
ocat
iona
l and
aca
dem
iced
ucat
ion.
Fur
ther
mor
e, n
ew S
ectio
n 11
7(b)
requ
ires
stat
es to
ado
pt o
ne o
f thr
ee m
easu
res
ofw
ork-
rela
ted
skill
s, in
clud
ing
atta
inm
ent o
fun
ders
tand
ing
of a
ll as
pect
s of
the
indu
stry
, as
wel
l as
a m
easu
re o
f acc
ess
and
equi
ty fo
r sp
ecia
lpo
pula
tions
.
5.3
fn
rn 00
(43
00
5 ti
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 118
. (b)
LO
CA
L P
RO
GR
AM
SEC
. 117
. (b)
LO
CA
L P
RO
GR
AM
As
an im
port
ant p
art o
f m
akin
g th
e an
nual
IMPR
OV
EM
EN
T P
LA
NB
egin
ning
not
less
IMPR
OV
EM
EN
T P
LA
NB
egin
ning
not
eval
uatio
n be
com
e pa
rt o
f an
ong
oing
pro
cess
than
1 y
ear
afte
r th
e im
plem
enta
tion
of th
ele
ss th
an 1
yea
r af
ter
the
impl
emen
tatio
n of
the
for
impr
ovin
g vo
catio
nal p
rogr
ams,
new
Sec
tion
prov
isio
ns o
f Se
ctio
n11
7, a
ll re
cipi
ents
of f
unds
prov
isio
ns o
f Se
ctio
n 11
5, if
any
rec
ipie
nt11
8(b)
req
uire
s th
atal
lrec
ipie
nts
of f
unds
und
erun
der
Par
t C o
f Titl
e II
will
dev
elop
a lo
cal
desc
ribe
d in
Sub
sect
ion
(a)
dete
rmin
es th
at th
ePa
rt C
of
Titl
e II
ann
ually
dev
elop
a lo
cal
prog
ram
impr
ovem
ent p
lan
that
sum
mar
izes
the
reci
pien
t is
not m
akin
g su
bsta
ntia
l pro
gres
s in
prog
ram
impr
ovem
ent p
lan
base
d on
the
annu
alse
lf-ev
alua
tion
cond
ucte
d un
der
Sub
sect
ion
(a).
mee
ting
the
stan
dard
s an
d m
easu
res
deve
lope
dev
alua
tion.
A r
ecip
ient
that
doe
s no
t mee
t the
sta
ndar
dsas
req
uire
d by
Sec
tion
115,
suc
h re
cipi
ent s
hall
unde
r S
ectio
n 11
7 m
ust d
escr
ibe
in it
s lo
cal
prog
ram
impr
ovem
ent p
lan
spec
ific
stra
tegi
esde
velo
p a
plan
, in
cons
ulta
tion
with
teac
hers
,pa
rent
s, a
nd s
tude
nts
conc
erne
d, f
or p
rogr
amfo
r m
akin
g su
bsta
ntia
l pro
gres
s (a
s de
fined
by
impr
ovem
ent f
or th
e su
ccee
ding
sch
ool y
ear.
the
Sta
te B
oard
und
er S
ectio
n 11
5(c)
) to
war
dSu
ch p
lan
shal
l des
crib
e ho
w th
e re
cipi
ent w
illth
ose
stan
dard
s in
the
com
ing
year
. Suc
h lo
cal
iden
tify
and
mod
ify
prog
ram
s fu
nded
und
erpr
ogra
m im
prov
emen
t pla
ns s
hall
be u
pdat
edPa
rt C
of
Titl
e II
, inc
ludi
ngan
nual
ly.
(1)
a de
scri
ptio
n of
voc
atio
nal e
duca
tion
and
care
er d
evel
opm
ent s
trat
egie
s de
sign
ed to
achi
eve
prog
ress
in im
prov
ing
the
effe
ctiv
enes
s of
the
prog
ram
con
duct
edw
ith a
ssis
tanc
e un
der
this
Act
; and
(2)
if n
eces
sary
, a d
escr
iptio
n of
str
ateg
ies
desi
gned
to im
prov
e su
pple
men
tary
serv
ices
pro
vide
d to
indi
vidu
als
who
arc
mem
bers
of
spec
ial p
opul
atio
ns.
5i
AzA cr .0
MN
MI
1111
1N
MI M
I MIN
MI
=II
UM
MS
MI M
I all
OM
NB
INS
NB
NB
IND
Ile N
MI
1111
1N
MI O
n.1
1111
111
1111
11N
W IN
N N
M
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 118
. (c)
ST
AT
E A
ND
LO
CA
L J
OIN
TSE
C. 1
17. (
c) S
TA
TE
AN
D L
OC
AL
JO
INT
As
unde
r th
e ol
d A
ct, n
ew S
ectio
n 11
8(c)
PLA
Nlf
, 1 y
ear
afte
r im
plem
enta
tion
of th
ePL
AN
-11,
aft
er 1
yea
r of
impl
emen
tatio
n of
requ
ires
that
a jo
int l
ocal
and
sta
te im
prov
emen
tpl
an d
escr
ibed
in S
ubse
ctio
n (b
), a
nd e
ach
year
the
plan
des
crib
ed in
Sub
sect
ion
(b),
suf
fici
ent
plan
be
deve
lope
d if
sub
stan
tial p
rogr
essa
sth
erea
fter
, sub
stan
tial p
rogr
ess
in m
eetin
g th
epr
ogre
ss in
mee
ting
the
stan
dard
s an
d m
easu
res
defi
ned
unde
r ne
w S
ectio
n 11
5(c)
in m
eetin
gst
anda
rds
and
mea
sure
s de
velo
ped
unde
rde
velo
ped
as r
equi
red
by S
ectio
n 11
5 ha
s no
tth
e st
anda
rds
and
mea
sure
s ha
s no
t bee
n m
ade
1Se
ctio
n 11
7 ha
s no
t bee
n m
ade,
the
Stat
e sh
all
been
mad
e, th
e St
ate
shal
l wor
k jo
intly
with
year
aft
er im
plem
enta
tion
of th
e fi
rst l
ocal
wor
k jo
intly
with
the
reci
pien
t, an
d te
ache
rs,
pare
nts,
and
stu
dent
s co
ncer
ned,
to d
evel
op a
the
reci
pien
t and
teac
hers
, par
ents
, and
stu
dent
sco
ncer
ned
to d
evel
op a
pla
n fo
r pr
ogra
mpr
ogra
m im
prov
emen
t pla
n de
velo
ped
unde
rne
w S
ectio
n 11
8(b)
, and
eac
h ye
ar th
erea
fter
.pl
an f
or p
rogr
am im
prov
emen
t. E
ach
such
pla
nim
prov
emen
t. E
ach
such
pla
n sh
all c
onta
insh
all c
onta
in(I
) a
desc
ript
ion
of th
e te
chni
cal a
ssis
tanc
e an
d(I
) a
desc
ript
ion
of th
e te
chni
cal a
ssis
tanc
e an
dpr
ogra
m a
ctiv
ities
the
Stat
e w
ill p
rovi
de to
prog
ram
act
iviti
es th
e St
ate
will
pro
vide
toen
hanc
e th
e pe
rfor
man
ce o
f th
e el
igib
leen
hanc
e th
e pe
rfor
man
ce o
f th
e re
cipi
ent;
reci
pien
t;(2
) a
reas
onab
le ti
met
able
to im
prov
e re
cipi
ent
(2)
a re
ason
able
tim
etab
le to
impr
ove
the
perf
orm
ance
und
er th
e pl
an;
scho
ol p
erfo
rman
ce u
nder
the
plan
;(3
) a
desc
ript
ion
of v
ocat
iona
l edu
catio
n(3
) a
desc
ript
ion
of v
ocat
iona
l edu
catio
nst
rate
gies
the
reci
pien
t will
ado
pt to
impr
ove
stra
tegi
es d
esig
ned
to im
prov
e th
epe
rfor
man
ce o
n th
e m
easu
res;
and
perf
onna
nce
of th
e pr
ogra
m a
s m
easu
red
(4)
if n
eces
sary
, a d
escr
iptio
n of
str
ateg
ies
the
by th
e ev
alua
tion;
and
reci
pien
t will
ado
pt to
impr
ove
(4)
if n
eces
sary
, a d
escr
iptio
n of
str
ateg
ies
supp
lem
enta
ry s
ervi
ces
prov
ided
tode
sign
ed to
impr
ove
supp
lem
enta
ryin
divi
dual
s w
ho a
re m
embe
rs o
f sp
ecia
lse
rvic
es p
rovi
ded
to in
divi
dual
s w
ho a
repo
pula
tions
.m
embe
rs o
f sp
ecia
l pop
ulat
ions
.
0
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 118
. (d)
FU
RT
HE
R A
CT
ION
The
Sta
tesh
all,
in c
onju
nctio
n w
ith th
e el
igib
le r
ecip
ient
,an
nual
ly r
evie
w a
nd r
evis
e th
e jo
int p
lan
deve
lope
dun
der
Subs
ectio
n (c
) in
ord
er to
impr
ove
prog
ram
perf
orm
ance
and
will
con
tinue
to d
o so
eac
hco
nsec
utiv
e ye
ar u
ntil
the
reci
pien
t sus
tain
s, f
or m
ore
than
1 y
ear,
sub
stan
tial p
rogr
ess
tow
ard
or m
eets
the
stan
dard
s de
velo
ped
unde
r Se
ctio
n 11
7.
SEC
. 119
. TE
CH
NIC
AL
ASS
IST
AN
CE
(a)
FED
ER
AL
ASS
IST
AN
CE
The
Sec
reta
ry s
hall
prov
ide
tech
nica
l ass
ista
nce
to th
e St
ates
, dir
ectly
or b
y gr
ant,
cont
ract
, or
coop
erat
ive
agre
emen
t,w
ith r
espe
ct to
the
deve
lopm
ent a
ndim
plem
enta
tion
of s
yste
ms
unde
r Se
ctio
n 11
5 an
dsu
bseq
uent
sec
tions
, inc
ludi
ng a
ssis
tanc
e on
(1)
setti
ng p
erfo
rman
ce s
tand
ards
;(2
) ev
alua
ting
the
relia
bilit
y, v
alid
ity, a
nd la
ck o
fbi
as o
f m
easu
res
and
stan
dard
s;(3
) se
lect
ing
and
deve
lopi
ng a
ppro
pria
teas
sess
men
t too
ls, i
nclu
ding
met
hods
for
asse
ssin
g ac
adem
ic g
ains
at t
he p
osts
econ
dary
leve
l;(4
) co
ordi
natin
g Pe
rkin
s m
easu
res
and
stan
dard
sw
ith o
ther
evo
lvin
g an
d em
ergi
ng f
eder
alin
itiat
ives
suc
h as
Nat
iona
l Ind
ustr
y Sk
illSt
anda
rds
and
Scho
ol-t
o-W
ork
perf
orm
ance
mea
sure
s;(5
) co
llect
ing
and
diss
emin
atin
g in
form
atio
n on
mod
els
for
usin
g pe
rfor
man
ce d
ata
for
prog
ram
impr
ovem
ent,
part
icul
arly
at t
he lo
cal l
evel
;an
d(6
) pl
anni
ng f
or a
nd p
rovi
ding
tech
nica
l ass
ista
nce
to lo
cal r
ecip
ient
s.In
pro
vidi
ng s
uch
assi
stan
ce, t
he S
ecre
tary
sha
llut
ilize
exi
stin
g re
sour
ces
in o
ther
Fed
eral
age
ncie
s.
SEC
. 117
. (d)
FU
RT
HE
R A
CT
ION
The
Stat
e sh
all,
in c
onju
nctio
n w
ith th
e el
igib
lere
cipi
ent,
annu
ally
rev
iew
and
rev
ise
the
join
tpl
an d
evel
oped
und
er S
ubse
ctio
n (c
) in
ord
er to
impr
ove
perf
orm
ance
and
will
con
tinue
to d
oso
eac
h co
nsec
utiv
e ye
ar u
ntil
the
reci
pien
tsu
stai
ns, f
or m
ore
than
1 y
ear,
ful
fillm
ent o
fth
e St
ate
and
loca
l sta
ndar
ds a
nd m
easu
res
deve
lope
d un
der
Sect
ion
115.
SEC
. 115
. (e)
TE
CH
NIC
AL
ASS
IST
AN
CE
The
Sec
reta
ry s
hall
prov
ide
tech
nica
l ass
ista
nce
to th
e St
ates
with
res
pect
to th
e de
velo
pmen
t of
syst
ems
unde
rSu
bsec
tion
(a).
In
prov
idin
g su
ch a
ssis
tanc
e,th
e Se
cret
ary
shal
l util
ize
exis
ting
reso
urce
s in
othe
r Fe
dera
l age
ncie
s.
New
Sec
tion
118(
d) r
equi
res
that
the
join
t sta
tean
d lo
cal
-ogr
am im
prov
emen
t pla
n be
rev
ised
annu
ally
unt
il th
e re
cipi
ent s
usta
ins,
for
mor
eth
an 1
yea
r, s
ubst
antia
l pro
gres
s to
war
d or
mee
ts th
e st
anda
rds
deve
lope
d un
der
new
Sect
ion
117.
New
Sec
tion
119
subs
tant
ially
exp
ands
the
tech
nica
l ass
ista
nce
requ
irem
ents
at b
oth
the
stat
e an
d fe
dera
l lev
els.
The
se r
equi
rem
ents
addr
ess
man
y of
the
shor
tcom
ings
in S
tate
impl
emen
tatio
n of
Per
kins
per
form
ance
mea
sure
s an
d st
anda
rds
foun
d by
Ste
cher
et a
l.(1
994)
in th
eir
inte
rim
rep
ort.
Spec
ific
ally
,un
der
new
Sec
tion
119(
a), t
he S
ecre
tary
mus
tas
sist
the
Stat
es w
ith th
e te
chni
cal q
ualit
y of
thei
r m
easu
res
and
stan
dard
s, th
e la
ck o
fap
prop
riat
e as
sess
men
t too
ls, c
oord
inat
ion
with
othe
r w
orkf
orce
pre
pare
dnes
s in
itiat
ives
, and
loca
l im
plem
enta
tion
of a
pro
gram
impr
ovem
ent p
roce
ss.
61
enZ rn
Mill
IMO
IMO
1111
1111
1M
IM
I N
MM
IN N
M M
I11
111N
MI
Ili M
I
41,
OM
=I
MI
NIB
MO
NM
I M
N N
MI
1111
111
r al
l=
la=
IIIN
N
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SE
C. 1
19. (
b) S
TA
TE
AS
SIS
TA
NC
E--
Eac
hN
ew S
ectio
n 11
9(b)
req
uire
s ea
ch S
tate
Boa
rdS
tate
Boa
rd s
hall
prov
ide
tech
nica
l ass
ista
nce,
dire
ctly
or
by g
rant
, con
trac
t, or
coo
pera
tive
to a
ssis
t loc
al r
ecip
ient
s w
ith s
ucce
ssfu
llyim
plem
entin
g a
prog
ram
impr
ovem
ent p
roce
ss.
agre
emen
t, to
loca
l rec
ipie
nts
with
res
pect
toFu
rthe
rmor
e, th
e 19
95 A
ct r
equi
res
that
Sta
tes
the
impl
emen
tatio
n of
Sec
tions
117
and
118
,
incl
udin
gse
t asi
de a
n an
nual
am
ount
of
Perk
ins
fund
ssp
ecif
ical
ly f
or p
rovi
ding
tech
nica
l ass
ista
nce
to
(1)
assi
stan
ce o
n ev
alua
ting
the
effe
ctiv
enes
s of
prog
ram
sba
sed
on th
e pe
rfor
man
cem
easu
res
and
stan
dard
s de
velo
ped
unde
r
loca
l rec
ipie
nts.
Sec
tion
117;
(2)
assi
stan
ce o
n id
entif
ying
and
ado
ptin
gap
prop
riate
str
ateg
ies
for
impr
ovin
gpe
rfor
man
ce o
n su
ch m
easu
res
and
stan
dard
s;(3
) tr
aini
ng lo
cal a
dmin
istr
ator
s an
din
stru
ctor
s in
the
use
of p
erfo
rman
ce d
ata
to im
prov
e vo
catio
nal c
ours
es a
ndpr
ogra
ms
base
d on
the
asse
ssm
ent
cond
ucte
d un
der
Sec
tion
117(
a); a
nd(4
) di
ssem
inat
ing
to lo
cal r
ecip
ient
s ex
ampl
esof
the
effe
ctiv
e us
e of
per
form
ance
dat
a to
impr
ove
voca
tiona
l cou
rses
and
pro
gram
s.S
ectio
n 10
2 al
loca
tes
an a
nnua
l am
ount
for
the
tech
nica
l ass
ista
nce
activ
ities
iden
tifie
d un
der
this
sub
sect
ion
from
the
allo
tmen
t mad
e to
each
Sta
te fr
om fu
nds
appr
opria
ted
unde
rS
ectio
n 3(
a).
663
it
1995
PE
RK
INS
AC
T19
90 P
ER
KIN
S A
CT
CO
MM
EN
TA
RY
SEC
. 120
. EV
AL
UA
TIN
G S
TA
TE
AN
DSE
C. 1
15. (
f) R
EPO
RT
The
Sec
reta
ry s
hall
New
Sec
tion
120
expa
nds
the
fede
ral e
valu
atio
n,N
AT
ION
AL
SY
STE
MS
subm
it a
repo
rtto
the
appr
opri
ate
com
mitt
ees
of P
erki
ns f
rom
a s
tric
t eva
luat
ion
of(a
)R
EPO
RT
The
Sec
reta
ry s
hall
subm
it a
of th
e C
ongr
ess
not l
ater
than
the
expi
ratio
n of
perf
orm
ance
mea
sure
s an
d st
anda
rds
to th
ere
port
to th
e ap
prop
riate
com
mitt
ees
of th
eth
e 4-
year
per
iod
begi
nnin
g on
the
date
of t
heen
tire
syst
em fo
r im
prov
ing
voca
tiona
lC
ongr
ess
not l
ater
than
the
expi
ratio
n of
the
enac
tmen
t of t
he C
arl D
. Per
kins
Voc
atio
nal
educ
atio
n as
intr
oduc
ed in
new
Sec
tion
115(
a)4-
year
per
iod
begi
nnin
g on
the
date
of t
heen
actm
ent o
f the
Car
l D. P
erki
ns V
ocat
iona
lan
d A
pplie
d T
echn
olog
y E
duca
tion
Act
Am
endm
ents
of 1
990.
Suc
h re
port
sha
llan
d de
taile
d in
sub
sequ
ent s
ectio
ns. H
owev
er,
new
Sec
tion
120
mai
ntai
ns a
n em
phas
is o
n th
ean
d A
pplie
d T
echn
olog
y E
duca
tion
Act
incl
ude
tech
nica
l qua
lity
of th
e co
re m
easu
res
and
Am
endm
ents
of19
95. S
uch
repo
rt s
hall
( I)
a d
etai
led
desc
riptio
n of
the
stat
us o
f eac
hst
anda
rds
impl
emen
ted
by th
e S
tate
s.in
clud
eS
tate
's s
yste
m o
f sta
ndar
ds a
nd m
easu
res
Add
ition
ally
, rat
her
than
em
phas
izin
g th
e(1
) a
deta
iled
desc
riptio
n of
the
stat
us o
fde
velo
ped
as r
equi
red
by th
is s
ectio
n;co
mpa
rabi
lity
of S
tate
-dev
elop
ed m
easu
res
and
each
Sta
te's
sys
tem
fori
mpr
ovin
gvo
catio
nal e
duca
tion
at th
e S
tate
and
(2)
an a
sses
smen
t of
the
valid
ity,
pred
ictiv
enes
s, a
nd r
elia
bilit
y of
suc
hst
anda
rds,
new
Sec
tion
120
requ
ires
the
Sec
reta
ry to
exa
min
e th
e ex
tent
to w
hich
the
loca
l lev
el d
evel
oped
as
requ
ired
byst
anda
rds
and
mea
sure
s, u
nbia
sed
to s
peci
alsy
stem
s re
quir
ed u
nder
the
prop
osed
new
Sec
tion
115
and
subs
eque
nt s
ectio
ns;
popu
latio
ns, i
n th
e ar
eas
of a
cade
mic
legi
slat
ion
have
act
ually
led
to p
rogr
am(2
) an
ass
essm
ent o
f the
rel
iabi
lity,
val
idity
,an
d la
ck o
f bia
s of
the
mea
sure
s an
dac
hiev
emen
t, vo
catio
nal s
kill
com
pete
ncie
s,em
ploy
men
t out
com
es, a
nd p
osts
econ
dary
impr
ovem
ent a
t the
loca
l and
Sta
te le
vels
.
stan
dard
s ad
opte
d un
der
Sec
tion
117;
cont
inua
tion
and
atta
inm
ent;
and
and
(3)
an e
valu
atio
n of
the
com
para
bilit
y of
Sta
te(3
)an
asse
ssm
ent o
f the
deg
ree
to w
hich
the
deve
lope
d pe
rfor
man
ce s
tand
ards
acro
ssus
e of
thes
e sy
stem
s ha
ve le
d to
pro
gram
impr
ovem
ent a
t the
Sta
te a
nd lo
cal l
evel
s.St
ates
to e
stab
lish
a co
re o
f co
mm
onin
dica
tors
.
646
5
to .0
UM
OS
NIS
SIN
NM
IN
NI
1E11
NM
IIN
N M
I IN
IN
N N
M11
111
NE
IIN
N B
P M
N N
I