wia performance and common measures where are we now? by anthony l. joseph, ph.d. program manager...

WIA Performance and Common Measures

Where are we now?

WIA Performance and Common Measures

Where are we now?

by

Anthony L. Joseph, Ph.D.Program Manager

Workforce Development & Training Division, NYSDOL

WIA Performance & Common Measures 2

Discussion OverviewDiscussion Overview

1. WIA Performance, the Basics• Perspectives on performance.

• How did New York State do.

• How did Local Areas do.

• Persistent Problems.

2. USDOL/ETA Common Measures Policy

• What do Common Measures mean?

3. Implementation of Common Measures

• USDOL & NYSDOL Expectations

• WIA Program Year (PY) 2005 & PY 2006 performance negotiations


References – USDOL/ETAReferences – USDOL/ETATraining and Employment Guidance Letter (TEGL) 28-04,“Common Measures Policy,” April 15, 2005 (Rescinds TEGL 15-03 “Common Measures Policy,” December 10, 2003).

TEGL 27-04,“Negotiating Workforce Investment Act (WIA) Title 1B Performance Goals for Program Years (PY) 2005 and 2006,” April 14, 2005.

Federal Register Notice (FRN) / Information Collection Request (ICR) “Workforce Investment Act Standardized Reporting Database (WIASRD) Specifications for Common Measures,” ICR, WIA Management Information and Reporting System, Vol.70, No.59, March 29, 2005.

FRN/ICR, “Proposed ICR Submitted for Public Comment and Recommendations; ETA Management Information and Longitudinal Evaluation (EMILE) Reporting System,” Vol. 69, No. 136, July 16, 2004.

TEGL No. 7-99, “Core and Customer Satisfaction Performance Measures for the Workforce Investment System,” March 3, 2000.


References – New York State References – New York State

WDS Technical Advisory (TA) #05-2, LWIA plans and strategies to comply with reporting requirements under the U.S. Department of Labor’s (USDOL) “Common Measures Policy” and “ETA Management Information and Longitudinal Evaluation (EMILE) Reporting System,” February 4, 2005.

NYSDOL Comments on EMILE Reporting System, September 10, 2004 – In response to USDOL/ETA’s Federal Register Notice of July 16, 2004 .

WDS TA #04-18, “Common Measures Youth Literacy and Numeracy Gains – New Requirement to Provide Guidelines on Entering Literacy and Numeracy Data into OSOS,” September 20, 2004.

WDS TA #04-5, “NYSDOL Policy Regarding the Implementation of Common Performance Measures for Workforce Investment Act Title 1B Employment and Training Programs,” May 7, 2004.


Performance PerspectivesPerformance Perspectives

1. USDOL/ETA perspective on program performance.

• WIA Performance Measures.

• Common Measures Policy.

2. Administration/Congress/Public and other stakeholder perspectives on program performance

• How many individuals (participants) are we serving.

This is as important as the USDOL/ETA performance measures … If not more important!


Comparison of WIA PY’04 Q3 to PY’03 Q3Participant and Exit Levels


Performance Items Program Group

PY'04 Q3 Performance

Level

PY'03 Performance

Level

PY'04Q3 as % of PY'03

Total Participants Adults 42,844 46,791 92% (July 2004 - March 2005) Dislocated Worker 28,466 33,673 85%

Older Youth 3,180 3,542 90%

Younger Youth 20,441 23,170 88%

Total Exiters Adults 19,122 31,855 60% (April 2004 - December 2004) Dislocated Worker 11,701 18,753 62%

Older Youth 1,626 2,116 77%

Younger Youth 8,199 11,179 73%




Total Participants New Registrants

ProgramPY'03

Q3PY '04

Q3 Change

Adult 39,378 42,844 3,466

Dislocated Wk. 29,201 28,466 -735

Older Youth 3,224 3,180 -44

Younger Youth 22,375 20,441 -1,934 1 Based upon PY 2003 WIASRD used for Annual Report.2 Based upon PY 2004 Qtr 3 WIASRD.

New Registrants ExitersPY'03

Q3PY '04

Q3 Change

22,222 21,527 -695

14,520 13,224 -1,296

1,691 1,507 -184

7,843 7,437 -406

ExitersPY'03

Q3PY '04

Q3 Change

26,590 19,122 -7,468

15,292 11,701 -3,591

1,781 1,626 -155

10,566 8,199 -2,367


Adult (18 and older)

1. Entered Employment

2. Employ. Retention

3. Avg. Earnings Change

4. Credential Attain.

Dislocated Worker





WIA Performance MeasuresPY 2004

WIA Performance MeasuresPY 2004

Older Youth (ages 19 to 21)





Younger Youth (ages 14 to 18)

1. Diploma/GED Attain.

2. Retention

3. Skill Attain.

Customer Satisfaction – Individuals and Business Customers


NY Statewide WIA Performance – PY’04 Quarter 3

NY Statewide WIA Performance – PY’04 Quarter 3

Performance Measure Program GroupPerformance

LevelPerformance

Index

Entered Employment Rate Adults 75.1 106% (October 1, 2003 - September 30, 2004) Dislocated Worker 77.6 97%

Older Youth 61.5 98%

Retention Rate Adults 84.1 105% (April 1, 2003 - March 31, 2004) Dislocated Worker 85.5 97%

Older Youth 72.6 94% Younger Youth 49.6 106%

Earnings Change/Replacement Adults 2,723 105% (April 1, 2003 - March 31, 2004) Dislocated Worker 86.0 99%

Older Youth 2,758 95%

Credential Rate Adults 76.4 125% (October 1, 2003 - June 30, 2004) Dislocated Worker 63.9 121%

Older Youth 52.6 122%

Diploma Attainment Rate Younger Youth 48.6 113% (April 1, 2004 - December 31, 2004)

Skill Attainment Rate Younger Youth 71.4 102% (April 1, 2004 - December 31, 2004)

Customer Satisfaction Employers 71.3 103% (January 1, 2004 - December 31, 2004) Participants 74.6 104%


Ent. Emp.

Retn. Rate

Ern. Chg.

Cred. Rate

Ent. Emp.

Retn. Rate

Ern. Chg.

Cred. Rate

Ent. Emp.

Retn. Rate

Ern. Chg.

Cred. Rate

Retn. Rate

Dipl. Attn. Rate

Skill Attn. Rate

PY'04 Q2 - - 15 1 - - - - 4 6 16 10 2 3 1

PY'03 - - 13 - - - 1 - 2 5 12 10 - 1 2

PY'02 - - 13 2 - - 1 2 2 4 17 13 1 9 5

PY'01 - - 13 8 2 - 2 8 14 10 16 22 7 16 9

Program Year / WIA

Perf. Measure

Adult Dislocated Worker Older Youth Younger Youth

A Look at Local Area PerformanceNumber of Locals Areas Failing

PerformancePY 2004 Quarter 2

A Look at Local Area PerformanceNumber of Locals Areas Failing

PerformancePY 2004 Quarter 2

Persistent Problem Measures:

• Earnings Change – Adult & Older Youth

• Credential Rate – Older Youth


Local Areas inSanction Status

"F" -- Failed the performance standard for the measure in the current quarter – PY’04 Q2.

"S1" Sanction Status Failed in 1 Year – PY’03.

(13 LWIAs)

"S2" Sanction Status Failed in 2 consecutive years – PY‘02&‘03.

(6 LWIAs)

"S3" Sanction Status Failed in 3 consecutive years – PY ‘01, ‘02 & ‘03.

(13 LWIAs)

Ent. Emp.

Retn. Rate

Ern. Chg.

Cred. Rate

Ent. Emp.

Retn. Rate

Ern. Repl.

Cred. Rate

Ent. Emp.

Retn. Rate

Ern. Chg.

Cred. Rate

Retn. Rate

Dipl. Attn. Rate

Skill Attn. Rate

New York State

Albany/Renss./Schen. F FAllegany/Cattaraugus FBroome/Tioga F FS2 S2 S3Cayuga/Cortland F FS3Chautauqua S3 F S3Chem./Schuyler/Steu. FS3 FChen./Del./Otesgo

Clint./Ess./Frank./Ham.

Columbia/Greene S2Dutchess F S2 S1Erie FS3 F FFulton/Montg./Scho. FGen./Liv./Orl./Wyom. FS3 S1 S3 S1Hempstead F F FJefferson/Lewis FMonroe FS3 S1 F S3 FNew York City F S2Niagara FS3 S1 FS3 FS3 F S3Oneida/Herk./Madison F F F FOnondaga F S3 S3Ont./Wayne/Sen./Yates FS3 S3Orange FS3 F F FS1 FOswego FOyster Bay FS3 S1 FS1 FRockland (no data for PY04Q2)

Saratoga/Warr./Wash. S1 F F FSt. Lawrence S1Suffolk FS3 FS1 FSullivan S1Tompkins S2 S2Ulster FS1 FS1Westchester S1 F FYonkers, City of F FS1 FS1 S2 F F

Adult Dislocated Worker Older Youth Younger Youth


USDOL/ETA Common MeasuresUSDOL/ETA Common Measures

• USDOL/ETA’s Common Measures Policy

TEGL No. 28-04 (rescinds TEGL 15-03, December 10, 2003)

• All programs administered by USDOL/ETA will be subject to the common measures – Covers:

1) Employment Service/Labor Exchange; 2) Trade Adjustment Assistance; and 3) WIA (including National Emergency Grants).

• No longer separate performance measures for the WIA Adult and Dislocated Worker programs, and the WIA Older Youth and Younger Youth programs.

Eligibility for these programs does not change.


USDOL/ETA Common Measures Policy(per TEGL 28-04)

USDOL/ETA Common Measures Policy(per TEGL 28-04)

Adult Measures:

1. Entered Employ. Rate

2. Retention Rate

3. Earnings Increase

Youth Measures:

1. Placement in Employment or Education.

2. Attain. of a Degree or Certificate

3. Literacy and Numeracy


Notable Changes Introduced in TEGL 28-04Notable Changes Introduced in TEGL 28-04

1. Efficiency measure removed.

2. Definition of Exit and Date of Exit.

3. Participants counted in performance.

4. Individuals in secondary school at exit are included in youth measures.

5. Supplemental employment information is valid for employment and retention outcomes.

6. Adult Earning Change measure methodology revised.

7. Literacy & Numeracy measure applies only to Out-of-School Youth.


Definition of ExitDefinition of Exit

Exit – Occurs when a participant does not receive a service funded by the program (or a partner program) for 90 consecutive calendar days, and is not scheduled for future services.

Date of Exit – Is applied retroactively to the last day on which the participant received a service funded by WIA or a partner program.

NYSDOL is considering modifying OSOS to:

• Produce automatic exits for participants.

• Allow services provided by a partner program to be used to extend the point of exit

This is allowed, provided the program has the capacity to track participants until the individual exits all services funded by the program or partner program.


Participants Counted in PerformanceParticipants Counted in Performance

Encourages inclusion of individuals who access core self-services and/or information activities in performance.

Furthermore, FRN/IRC (March 29, 2005) is calling for reporting of customer detail on these individuals in the WIASRD specification.

These individuals are currently excluded from performance measurement. However, the Administration is proposing to remove this exemption as part of WIA reauthorization.

NYSDOL is assessing reporting this information from Local area swipe card data.


Youth in Secondary School at Exit Youth in Secondary School at Exit

Youth in secondary school at exit are included in the youth measures:

Placement in Employment or Education

Attainment of a Degree or Certificate

TEGL 15-03 excluded youth in secondary school at exit from these measures.

The rationale/motive is to encourage youth to successfully complete secondary school (contribute to better long-term success in the workforce).


Adult & Dislocated WorkerComparison of WIA and Common Measures


Adult Common Measures

Adult Dislocated Worker (DW) (Adult & DW)

Entered Employment

TEGL 7-99 TEGL 7-99 TEGL 28-04

Numerator Employed in 1st qtr. after exit Employed in 1st qtr. after exit Same as TEGL 7-99

Denominator Unemployed at registration All Dislocated Workers Same as TEGL 7-99

Retention TEGL 7-99 TEGL 7-99 TEGL 28-04

Numerator Employed in 3rd qtr. after exit Employed in 3rd qtr. after exit# employed in both the 2nd and 3rd qtrs. after exit.

Denominator Employed in 1st qtr. after exit Employed in 1st qtr. after exit Employed in 1st qtr. after exit

Program / Perf. Measures

WIA Performance Measures




Adult Common Measures

Adult Dislocated Worker (DW) (Adult & DW)

Credential TEGL 7-99 TEGL 7-99

NumeratorReceive credential by end of 3rd

qtr. after exit.Receive credential by end of 3rd

qtr. after exit.

DenominatorReceived training AND

employed in 1st qtr. after exit.Received training AND

employed in 1st qtr. after exit.

Earnings TEGL 7-99 TEGL 7-99 TEGL 28-04

Numerator -- Those employed in the 1st qtr. after exit

Earnings in 2nd & 3rd qtrs. After exit MINUS earnings in 2nd and 3rd qtrs. prior to registration.

Earnings in 2nd and 3rd qtrs. prior to dislocation.

Same as TEGL 7-99, Adult Earnings Change Measure - Earnings in 2nd & 3rd qtrs. After exit MINUS earnings in 2nd and 3rd qtrs. prior to registration.

Denominator -- Those employed in the 1st qtr. after exit

# employed in 1st qtr. after exit.

Earnings in 2nd and 3rd qtrs. after exit.

Same as TEGL 7-99, Adult Earnings Change Measure - # employed in 1st qtr. after exit.




Youth (Older & Younger)Comparison of WIA and Common Measures


Youth Common Measures

Older Youth Younger Youth (Older & Younger Youth)Entered Employment & Placement in

TEGL 7-99 - Entered Employment Only TEGL 28-04

Numerator Employed in 1st qtr. after exit

# in employment, military, post-secondary education and/or adv. training/occupational skills training in the 1st qtr. after exit.

Denominator Unemployed at registration # who exit during the quarter

Retention TEGL 7-99 TEGL 7-99

Numerator Employed in 3rd qtr. after exit# engaged in 1of 5 retention activities in 3rd qtr. after exit.

Denominator Employed in 1st qtr. after exit All exiters

Earnings TEGL 7-99

Numerator

Earnings in 2nd & 3rd qtrs. After exit MINUS earnings in 2nd and 3rd qtrs. prior to registration.

Denominator # employed in 1st qtr. after exit.







Older Youth Younger Youth (Older & Younger Youth)

TEGL 7-99 TEGL 7-99 TEGL 28-04

Credential / Diploma / GED Diploma / GED Degree / Certificate

NumeratorReceive credential by end of 3rd qtr. after exit.

Attain a diploma/GED by the end of the 1st qtr after exit.

# who attain a diploma, GED, or certificate by the end of the 3rd qtr. after exit.

Denominator# employed, in post-secondary education or in advanced training in the 1st qtr. after exit.

No diploma/GED at registration.# enrolled in education (at registration or at any point during the program) who exit during the qtr.

Skill Attainment TEGL 7-99Numerator # of skill goals attainedDenominator # of skill goals set

Credential / Diploma / GED / Certificate




Youth (older & Younger)Comparison of WIA and Common Measures

Youth (older & Younger)Comparison of WIA and Common Measures

• Applies to Out-of-school youth only.

• Pretest must occur 60-days following enrollment.

• NYSDOL will require Local areas to report this information on a quarterly basis beginning PY’05.


Older Youth Younger Youth (Older & Younger Youth)

Literacy and Numeracy TEGL 28-04

Numerator# who increase one or more educational functioning levels.

Denominator

Of those determined to be basic skills deficient at pre-test, the # who have completed 1-year in the program PLUS the # who exit before completing 1-year in the program.




Negotiating Performance for PY 2005 and PY 2006

Negotiating Performance for PY 2005 and PY 2006Program / Performance Measures Source of Definition

Adult / Dislocated Worker Measures

Entered Employment Rate Common Measures Policy (same as TEGL 7-99)

Employment Retention Rate Common Measures Policy

Earnings ChangeCommon Measures Policy (same as TEGL 7-99 Adult Earning Change)

Employment and Credential Rate TEGL 7-99

Customer Satisfaction (Emp. & Job Seeker) TEGL 7-99

Older Youth Measures

Entered Employment Rate TEGL 7-99

Employment Retention Rate TEGL 7-99

Earnings Change TEGL 7-99

Credential Rate TEGL 7-99

Younger Youth Measures

Skill Attainment Rate TEGL 7-99

Diploma or Equivalent Attainment Rate TEGL 7-99

Retention Rate TEGL 7-99


Performance Measure CohortsPY’04, ‘05 & ‘06

Performance Measure CohortsPY’04, ‘05 & ‘06

Program Year 2004 Program Year 2005 Program Year 2006

Performance Measure / Quarter

Apr - Jun 2003

Jul - Sep 2003

Oct - Dec 2003

Jan - Mar 2004

Apr - Jun 2004

Jul - Sep 2004

Oct - Dec 2004

Jan - Mar 2005

Apr - Jun 2005

Jul - Sep 2005

Oct - Dec 2005

Jan - Mar 2006

Apr - Jun 2006

Jul - Sep 2006

Oct - Dec 2006

Jan - Mar 2007

Apr - Jun 2007

Total ParticipantsAdult, DW, OY, YY

Adult Exiters

DW Exiters

Older Youth Exiters

Younger Youth Exiters

Diploma Attainment - YY

Skill Attainment - YY

Entered EmploymentAdult, DW, OY

Credential - Adult, DW, OY

Retention - Adult, DW, OY, YY

Earnings - Adult, DW, OY


The End

Thank you!

The End

Thank you!

wia performance and common measures where are we now? by anthony l. joseph, ph.d. program manager...

Documents

usdoleta performance

wia performance measures

program performance

performance perspectives

common measures youth

performance negotiations

b performance goals

wia management information