the des performance framework - … · red tape and administration kpi 2.8 and ongoing support...

The DES Performance Framework

An issues paper presented by Disability Employment Australia November 2012

Issues Paper DES Performance Framework Review 12 November 2012

2

Contents In Brief ....................................................................................................................................... 3

Consultation Process ............................................................................................................. 3

Summary of Recommendations ............................................................................................ 3

Key Issues .............................................................................................................................. 3

KPI 2.8 and Ongoing Support............................................................................................. 3

Red tape and administration ............................................................................................. 4

Attachment A – Consultation Process ....................................................................................... 5

Reference Group ................................................................................................................... 5

Open Teleconference Consultations ..................................................................................... 5

Roundtable(s) with National Disability Services .................................................................... 5

Jobs Australia National Conference ...................................................................................... 5

Expert Reference Group ........................................................................................................ 5

Board Endorsement ............................................................................................................... 5

Attachment B – Summary of Recommendations ...................................................................... 6

Attachment C – KPI 2.8 and Ongoing Support .......................................................................... 7

Introduction ........................................................................................................................... 7

KPI 2.8 .................................................................................................................................... 7

Ongoing Support Proposals from DESPFRWG members ....................................................... 7

Our Principles/Objectives ...................................................................................................... 8

Real and/or perceived risks to performance ......................................................................... 9

Real and/or perceived undesirable behaviours .................................................................... 9

Appendix A – KPI 2.8 does not drive continuous improvement ..................................... 10

Conclusions .......................................................................................................................... 12

Attachment D – Alternate Handling of Pathway Outcomes ................................................... 13

Combining 13/26 Week Measures ...................................................................................... 13

Pathway Proposal ............................................................................................................ 13

Discussion ........................................................................................................................ 14

Recommendations ........................................................................................................... 15

Bonus proposal for discussion ......................................................................................... 15

Attachment E – 52 Week Outcomes / Sustainability Indicator ............................................... 16

Sustainability Indicator (52 Week Outcome Measure) ....................................................... 16

For discussion ...................................................................................................................... 17

Attachment A – Sustainability indicator denominator .................................................... 19


3

In Brief Disability Employment Australia commends the Minister and the Department for reviewing the DES Performance Framework. Our consultation process has been enhanced through decisions that enabled papers to be unclassified. This paper sets out to provide a briefing of our consultation process, our recommendations and the key issues that have arisen.

Consultation Process

We conducted a comprehensive and collaborative consultation process from August to October 2012, including the formation of a Performance Framework Taskforce, plus an Expert Reference Group. With National Disability Services, we hosted three roundtables for DES providers: in Sydney, Perth and Melbourne. We also consulted when invited to present at the Jobs Australia national conference. The full details of our consultation process are included in Attachment A.

Summary of Recommendations

Disability Employment Australia believes the enhancements recommended by the DES Performance Framework Review Working Group (DESPFRWG) regarding the DES Performance Framework will lead to a greater focus on quality job matches that lead to sustainable outcomes. The enhancements are also likely to mitigate alleged program churn rates. Our issues and consultative feedback have been carefully considered by DEEWR and it is clear we have been listened to, and strongly influenced the process. A summary of our recommendations has been included in Attachment B. Nonetheless, there are two issues in particular that Disability Employment Australia believes should be clarified for our members, and require further consideration and response from DEEWR.

Key Issues

Perverse operation of KPI2.8 and measuring Ongoing Support quantitatively

Red tape and administration

KPI 2.8 and Ongoing Support Disability Employment Australia is not recommending that Ongoing Support is removed from the framework. The provision of Ongoing Support is coherently important to DES program objectives. Rather, our consultation process has questioned the operation of its measurement in KPI2.8. This process has highlighted the real and/or perceived risks to performance and the real and/or perceived perverse behaviours as a result of this measurement. There appears to be consensus that KPI 2.8 contains perversities. Disability Employment Australia submitted a solution: temporarily apply a 0% weighting or move Ongoing Support to KPI3 and measure qualitatively. However, it was rejected by DEEWR as not being a good solution, on the grounds that the measurement of Ongoing Support is integral to the DES program: especially DES-ESS. The three objectives of the DES KPI measures are:

To drive continuous improvement; and

To inform purchasing (for largely the same reason); and

To facilitate participant choice.


4

Attachment C contains our logically cogent argument demonstrating how KPI 2.8 does not drive continuous improvement and contains perversities contrary to program objectives. Therefore in this matter we believe the burden of proof lies with DEEWR.

To demonstrate how the quantitative measurement of Ongoing Support is integral to the DES program; and

To show why a measure known to be perverse has a 15% performance weighting in DES-ESS and 5% in DES-DMS.

Given that Ongoing Support is the cornerstone of Disability Employment Services attempts should be made to reconcile its importance and fair and reasonable application of the measure to benefit all stakeholders: participants, employers, providers, DEEWR and taxpayers.

Red tape and administration Disability Employment Australia is concerned this review process has introduced unnecessary complexity, administration and red tape. Complexity has been added to the calculation of KPI measures. New guidelines will be required and two additional verifications are required. Complexity, administration and red tape divert resources and time away from participants because they result in increased costs.

New verifications required at 52 Weeks and then 6 weeks following Independent Worker exit

Preference for written verification from the employer and the worker o This raises disclosure issues; privacy issues; policy implications, etc. o Substantially more costly than allowing verification with by phone and/or

with one party

We predict the vast majority of exits will occur at the 52 Week Sustainability Indicator point

o For many, this effectively creates a 58 Week Sustainability Indicator that needs to verified in writing by the employer and worker twice within a six week period


5

Attachment A – Consultation Process

Reference Group

Disability Employment Australia has convened a group of members to provide strategic advice and subject matter expertise in order to inform our recommendations to DEEWR's DESPFRWG with the prime objective of improving quality outcomes for end users (participants and employers). To date we have had five teleconferences:

2 August 2012

16 August 2012

13 September 2012

28 September 2012

05 October 2012

Open Teleconference Consultations

We have hosted a number of teleconferences for members. These sessions provided an interactive forum for members to provide feedback regarding our discussion paper and the topics selected for the DEEWR DES Performance Framework Review Working Group meetings. Teleconferences were scheduled for the following days:

20 August 2012 (4 x Teleconferences)

08 October 2012 (2 x Teleconferences)

Roundtable(s) with National Disability Services In order to provide the greater opportunity for member input to the performance review process Disability Employment Australia held joint consultation process with National Disability Services.

Sydney 28 August 2012

Perth 10 September 2012

Melbourne 21 September 2012

Jobs Australia National Conference Jobs Australia invited Disability Employment Australia to present on the DES Performance Framework at their annual conference. Approximately fifty delegates attended this session.

Sydney 31 October 2012

Expert Reference Group

Disability Employment Australia has consulted with experts in the fields of public policy and mathematics throughout the review. This group has met frequently and informally.

Board Endorsement

The Disability Employment Australia board endorsed our consultation process and recommendations. Two board members sit on our Expert Reference Group and broader Reference Group.

Adelaide 15 October 2012


6

Attachment B – Summary of Recommendations Topic Our Recommendations Current Status

Commencement to referral ratio (KPI 1.1)

Remove from framework; contract management activity


Time taken to achieve 13 week employment outcome (KPI 1.2)


Remove from framework; auto-commencement facility

Alternate handling of 13/26 Week Outcomes

Did not support; adds complexity; more contracts at 1 and 2 stars. Pathway and full should be claimable in same period of service (Attachment D). Remove 8 hour BMs from pathway denominator.

Outcomes will be collapsed to reduce disproportionate impact of pathway outcomes. 8 hour BMs removed from pathway denominator.

Ongoing Support Apply a 0% weighting or move to KPI3 to address known perversities with measurement.

Maintain current measure and weighting; reduce risks by excluding short-term support from the measure (< 6 months). Verify Independent Worker Exits.

52 Week Sustainability Indicator

Prefer paid 52 week outcome with new funding. Supported unpaid for ESS but not for DMS. Supported 15% weighting if Ongoing Support proposal was accepted. Rename from outcome to indicator. Populate denominator with employment anchors only (Attachment E).

Renamed sustainability indicator; denominator only populated by employment anchors. Unpaid in both DMS and ESS with 10% weighting.

Job in Jeopardy

Supported inclusion of Job in Jeopardy in framework in 52WSI denominator. Did not support inclusion in Ongoing Support measure. Rename program.

Job in Jeopardy included in 52WSI measurement with 10% weighting. Program to be renamed.

Performance measures and weightings

Did not support weightings proposed based on concerns regarding operation of Ongoing Support measure.

Option 1 is likely with 10% on 52WSI and 15% (5%) on Ongoing Support. Other measures retain current weighting.

Hours and wages measure

Support in principle but also recognize the difficulties in measuring. Support for NDS pilot. Prefer an hours measure as wages follow hours.

NDS pilot will go ahead, findings will inform whether or not hours/wages included.

KPI3 Quality Support deferring issues to industry over the next six months.

Issues deferred to industry over the next six months.


7

Attachment C – KPI 2.8 and Ongoing Support

Introduction

We think we can all agree that what matters most is that we have a measurement system that rewards us for what we’re here to do and doesn’t punish us for the same.

We need to beware of measuring something that providers can gate keep. Measuring ongoing support directly creates an inherent risk.

Remove the risk of providing ongoing support and people will get the support they need to stay in employment.

It has been suggested that the best way to stop the risk in KPI2.8 is to put everyone who gets a 26-week outcome into the measure.

o But then, you are no longer measuring OS. You’re measuring the 52-week indicator.

If a person needs ongoing support then the goal of a 52-week sustainability indicator means ongoing support will be used to achieve it.

Further to this, there is a financial incentive and no disincentive to provide ongoing support for as long as required. The Ongoing Support Assessment determines support level after 52 weeks.

While some participants will require it in perpetuity, Ongoing Support should not be a destination; the client base is too big and diverse following uncapping.

The return to service rate suggests we can strike a better balance between throughput and time-unlimited support.

In our view, the DES program is about empowering self-sufficiency and independence.

KPI 2.8

Every DES KPI measure, except for KPI 2.8, requires a provider to do something in order to get recognition (in simple terms, get jobs).

By contrast, KPI 2.8 starts by giving full recognition (100%) and counts down but can be maintained by doing nothing.

Therefore, KPI 2.8 does not drive continuous improvement: a stated objective of the measure.

Ongoing Support Proposals from DESPFRWG members

DEA has submitted a strong case that KPI 2.8 is not fit for purpose and can drive perverse incentives incongruent with program objectives.


8

There appears to be consensus over this conclusion, but contest over the best way to address the problem: ultimately this falls within DEEWR’s purview.

We strongly believe that moving OS to KPI3 or giving it a 0% weighting will result in reduced return to service rates and sustainable outcomes.

In contrast to other suggestions, our recommendation does not add layers of complexity.

We believe a better balance can be struck between throughput and time-unlimited support.

This is evident in the declining usage of OS and the return to service rate.

Star ratings are good at assessing milestones and goals – but Ongoing Support does not have milestones.

Ongoing Support itself is neither a milestone nor a goal; rather it is a service available to meet program objectives.

Ongoing Support is an individual and personalised service and we should be learning by process benchmarking what strategies achieve independence efficiently and effectively – these are properties open to audit but not to star ratings.

Ongoing support should not be a numbers game.

Independence through socio-economic participation and inclusion are arguably the desirable goals, notwithstanding that Ongoing Support can and should be available on a time-unlimited basis for those who need it.

Support should be withdrawn as employers and natural supporters are empowered by DES to support the participant to be fully included in the workplace independent of Government funded assistance – but support should be reactivated at any time by employer or employee as required for the employment life of the individual.

Our Principles/Objectives

Ongoing support in the workplace for as long as required

People with disability have same opportunities as people without disability in

exercising choice and control over their careers

Quality job placements result in longer-term outcomes

Ongoing support is a feature of DES

Sustainable employment is the objective of DES

Ensure no disability group is advantaged or disadvantaged by this measure (or

provider by specialty/generalist)

DEEWR’s stated objective in this review was to examine the appropriateness of the current KPI 2.8, with particular regard for the handling of independent workers. Our consultation process suggests that the current measure of Ongoing Support (KPI 2.8) is not appropriate due to real/perceived risks.


9

Real and/or perceived risks to performance

It is virtually impossible to improve performance (see Appendix A)

Volatility of small caseloads (e.g. 1/1 = 100% but 1/2 = 50%)

Narrow and prescriptive program settings regarding permissible breaks

o Settings prioritise work in one job over ongoing employment, reportedly

reducing the effectiveness of Ongoing Support in the long-term

o On DEEWR’s evidence, 60% of those working after DES are in casual, part-

time or temporary employment (insecure work)1

o In today’s labour market, where up to 40% of work is insecure2, support

must be withdrawn if alternate employment can’t be found within 5 days

Real and/or perceived undesirable behaviours

Inappropriate usage through risk selection (i.e. rewarded for providing Ongoing

Support to those who don’t need it, and punished for providing it to those who do

need it – creaming and/or excluding)

Inappropriately holding placements beyond requirement

Early exits because independent worker exits remain in perpetuity

Allowing jobs to fall over at six months because there’s a greater financial and

performance incentive if the participant commences a new period of service

o Reduced incentive to use Job in Jeopardy

o Reduced incentive to use Ongoing Support

Manipulation of score by not providing the service (e.g. 1/1 = 100%)

The perverse consequences of risk selection and KPI 2.8 works against the evidence based principle of person-centered, time unlimited support. This results in a return to service rate that could be mitigated by removing disincentives to provide Ongoing Support, resulting in more sustainable employment outcomes and better value-for-money.

1 DEEWR, Labour Market Assistance Outcomes December 2011

2 ABS, Forms of Employment, Nov 2010, Cat 6539.0


10

Appendix A – KPI 2.8 does not drive continuous improvement

To improve performance in KPI 2.8 a provider must monitor their score in the measure. Based on its current (and proposed) operation, in order to calculate the number of replacements required for one participant commenced in Ongoing Support who is exited due to losing their job the following formula can be used.

In this example, a provider has a numerator of 9 and a denominator at 10. That would put their Ongoing Support score at 90%. One of the participants in the OS phase resigns and was not able to commence another one job within five days. The provider exits the participant from Ongoing Support. This takes the numerator and denominator to 8/10 and the score to 80%. How does the provider get back to 90%? By using the above formula:

The provider therefore needs ten independent worker exits (with an initial Ongoing Support fee) in order to improve their numerator and denominator to bring their Ongoing Support score back to 90%.

In this case, for just one fall out, the provider needs ten people to commence in Ongoing Support (with independent worker exits) to improve their performance. Let’s now change the target percentage to be that of the national average of Ongoing Support3 in DES-ESS (for the sake of argument) to determine the number of independent

3 National averages as at 21/09/12


11

worker exits required to compensate for one ‘loss of job’ exit in order to maintain national average performance.

Let’s call that 4. Now let’s assume the provider strives to achieve a four star rating and aims to be 20% above the national average for Ongoing Support performance.

The examples so far have concentrated on situations where the provider has had one ‘loss of job’ exit for an Ongoing Support participant for whom they have taken the first fee. The following example explores a more common situation in the DES sector in late 2012. Let’s assume that a large contract got off to a poor start and its Ongoing Support performance is now the same as the national average at 73.6%. The numerator and denominator would be 110.4 (make it 110) and 150 respectively.

Let’s call that 22 and continue to the use the earlier formula. In this case, we don’t just want to compensate for a loss of job exit; we want to ensure that the score in Ongoing Support increases. To do that:

In this common example, the provider needs 189 participants to commence in Ongoing Support (with independent worker exits) to bring their Ongoing Support score twenty per cent above national average. That’s more than their entire Ongoing Support caseload. Here’s how it works:

Consequently, it would be understandable that providers in a competitive quasi-market would adopt a risk management strategy to KPI 2.8. This risk management strategy should not be seen as ‘threatening the integrity of the program’. However, given that Ongoing Support is the cornerstone of Disability Employment Services (particularly DES-ESS) attempts should be made to reconcile its importance and fair and reasonable application of the measure to benefit all stakeholders: participants, employers, providers, DEEWR and taxpayers. Anecdotal feedback from members indicates that Ongoing Support usage is low due to the perceived threat to performance. This is exacerbated by the tiny window of five days permitted to find alternative employment following resignation. Any other loss of job requires an immediate removal of service.


12

Conclusions Disability Employment Australia agrees that Ongoing Support should be measured, monitored and reported on. However, we do not agree that it should be measured in a quantitative way with a performance weighting unless there is evidence that there are no perverse or unintended negative drivers to the delivery of quality Ongoing Support. To further illustrate our thinking regarding whether Ongoing Support should be a qualitative or quantitative measure, we refer to the DES-ESS selection criteria:

Your organisation’s understanding of the performance factors that lead to high quality sustainable Employment Outcomes for people with permanent disability, including the strategies your organisation will use with available resources to achieve:

sustainable Employment Outcomes

effective Interventions

Training and skills development

mentoring, and

Ongoing Support (likely to be on a regular and long-term basis).4

Ongoing Support can therefore be understood to be a performance factor that influences the sustainability and quality of employment outcomes. Ongoing Support is an individualised and personalised service; it is insufficient to measure Ongoing Support by who remains in one job over the long-term. Disability Employment Australia remains concerned that the proposals for the measurement of Ongoing Support do not remove the real and/or perceived risks to performance, although we do acknowledge the proposal delays the risk selection from 26 weeks to 52 weeks. We remain concerned that the measurement is inherently flawed and works against program objectives.

4 DEEWR, Request for Tender DES-ESS, Page 87


13

Attachment D – Alternate Handling of Pathway Outcomes

Combining 13/26 Week Measures

Disability Employment Australia does not prefer Option 1 because it adds complexity to an already contentious and poorly understood system;

Disability Employment Australia proposes an alternative option for the handling of pathway outcomes; and

Disability Employment Australia is currently consulting over the operation of the bonus outcome measure and will make recommendations to the DESPFRWG by 25 October 2012

Pathway Proposal We propose that program settings are changed to allow providers to claim both a pathway and full 13-week outcome in the same period of service. This can be done in a cost-neutral manner by making the full outcome payment a pro-rata payment if the pathway outcome had already been claimed. The functionality already exists in the Employment Services System through the ‘zero’ dollar claim. In this way, a provider can elect to spread the funding they would receive for a full 13 week outcome over a 26 week period. It is predicted that more employment and education opportunities will be offered to participants, as opposed to fortnightly appointments, or job clubs. This is because providers could achieve additional performance and financial rewards above and beyond quarterly service fees. We submit this will result in greater numbers of participants being engaged in capacity building activities, as demonstrated in the following table.

2nd

Service Fee 13 Week Pathway 13 Week Full (Pro-rata) 13 Week Full

Level 1 ESS $890 or $945 + 5% stars $1915 + 20% stars $2860

Level 1 DMS $1595 or $945 + 5% stars $1915 + 25% stars $2860

Level 2 ESS $1900 or $1450 + 5% stars $2950 + 20% stars $4400

Disability employment practice (or the open employment ‘place then train’ model) tells us that a principal focus on starting competitive employment sooner rather than later is more effective than traditional employment approaches. For example, traditional approaches include ‘train then place’, vocational counseling, sheltered employment or job clubs. The evidenced-based model is clear: assessment is best conducted as a continuous process of experience(s) in competitive employment. Evidence also shows that people who secure jobs that reflect their own choices remain in their jobs twice as long as when jobs are not aligned with their preferences. Therefore, it follows that:

Promoting decision-making and choice will lead to more sustainable employment outcomes; and

Employment sustainability is predicated upon a quality job match, informed through reflection on previous employment experiences.


14

We submit that allowing providers to claim both a pathway and full 13-week outcome in the same period of service will diversify service offerings that better meet evidence based principles of disability employment. The principle of jobs as transitions, where all jobs are seen as worthwhile, positive experiences – and do not represent failures to achieve optimal outcomes. But do lead, through continued disability employment assistance, to competitively obtained, permanent employment. It is further anticipated that acceptance of our proposal will result in a higher retention rate to optimal outcomes, because the best way of determining a persons’ capacity to work is for them to choose, get and keep employment and to reflect on those experiences. Changing program settings in this way provides greater scope for a participant to exercise choice and control over their vocational pathway by leaving the provider and participant with the opportunity to achieve an optimal outcome in the same period of service. The current framework limits choice by valuing a single job over employment, as providers perceive a disincentive to offer participants any opportunity that is not the optimal opportunity. The imperative to achieve full outcomes over pathway outcomes encourages risk selection: efforts are focused on those most likely to achieve full outcomes, and some participants are parked. Risk selection is likely to be mitigated by rewarding interim milestones which increase capacity of the part of participants to achieve full outcomes. The proposal is cost-neutral and better caters for individualised job search. Our members report significant numbers of participants who disagree with their employment benchmarks. Instead of focusing efforts on obtaining a reassessment, most of which do not result in reduced benchmarks, an incentive is created for an employment or education based activity – as opposed to the common alternatives of group sessions or fruitless fortnightly contacts. Claims that providers already do this are countered by the low pathway outcome rate and the fact that nearly 60% of DES participants achieve neither a full nor pathway outcome. It is widely accepted that performance weightings drive behavior more than financial incentives in the DES quasi-market. Therefore, it follows that if the performance disincentives to planned pathways were replaced with incentives – in a cost neutral way for Government – overall program effectiveness would increase by virtue of an increased and sustained outcome rate. More, it would better align the program with evidence based disability employment principles, and the economic and social inclusion of participants.

Discussion

If employment benchmarks were co-produced, would we need to differentiate between outcomes?

Should this proposal only apply to employment pathway outcomes?

Would increasing the pathway outcome rate reduce the disproportionate effect of pathway outcomes?

Will regression favour the fastest achievement of full outcomes?

DEEWR has indicated providers can already move between pathways and full outcomes, therefore:

o What proportion/number of participants achieve a pathway 13 week outcome for employment and then go to achieve a full 26 week outcome for employment in the same period of service?


15

o What proportion/number of participants achieve a pathway 13 week outcome for education and then go on to achieve a full 26 week outcome for employment in the same period of service?

o What does this breakdown look like across program, disability type, age and gender?

Recommendations

Defer outcome policy discussion to the DES Operational Working Group to respond to contentious and pressing industry feedback, for example:

o The non-payable outcome guidelines work against evidence-based disability employment principles

o How might pathway outcomes be universal to all participants? Possibly remove 8-hour benchmarks from pathway denominator

o Why is temporary work a non-payable pathway outcome, when it represents progress towards sustainable employment?

o What is the role of Intermediate Labour Market Programs? o What is the role of Social Enterprises? o Implications for re-anchoring to go for full 13 weeks? o Could an employment pathway be payable for a person with an 8-hour

benchmark?

Bonus proposal for discussion One of the key goals of uncapping DES was to engage people on DSP who were outside of the labour force. The participation of people with disability in the labour force – and their carers – is essential for the future funding of the NDIS. Recommendation: A non-payable bonus is triggered when an outcome is achieved for a directly registered DSP recipient by the provider who conducted the direct registration. Impact: Providers perceive an incentive to engage those outside of the labour force and to promote Disability Employment Services. Awareness of the program increases as a result.


16

Attachment E – 52 Week Outcomes / Sustainability Indicator

Sustainability Indicator (52 Week Outcome Measure)

Conditional support for an unpaid 52 week proxy indicator of sustainability (see

below)

Conditional support for automatic verification of sustainability indicator (see below)

DEEWR has stated its objective for introducing a 52 week outcome measure is to provide an

additional and specific incentive to DES providers to place participants into long-term

sustainable employment (beyond the 26 week outcome), and also to create a disincentive

for the possible churning of participants post the 26 week outcome.5

Disability Employment Australia is on record as recommending longer-term outcomes to

better incorporate quality into the DES Performance Framework. The focus on better

ensuring sustainable outcomes is clear in the DESPFRWG Terms of Reference:

The Performance Framework is to be reviewed to ensure it continues to provide

accountability and transparency, directly relates to the quality of services and

provides assistance to participants to secure sustainable employment (leading

to greater social inclusions).

Overwhelmingly, our members would like to see the introduction of a paid 52 Week

Outcome, with additional funding and indexation of fees. However, we agree it is a

contestable point and will continue to contest it moving into future contracting periods.

Members are concerned by the potential impact on viability of spreading the existing

funding envelope for a paid 52 week outcome in the contracting period, particularly in the

absence of any funding increase since 2006. Therefore, the option to include an unpaid 52

week proxy indicator has been explored in depth with the membership. Disability

Employment Australia has also consulted jointly with National Disability Services members.

The issues have been problematised by the Department’s decision to not include outcome

policies in the scope of the terms of reference for the review and referring to the proxy

measure as an outcome. Consequently, there is speculative anxiety regarding future

program settings that might make any agreements made without prior knowledge

untenable.

In our view, there is a strong case to include 52 weeks (as an unpaid proxy measure) in the

DES-ESS program only. We would prefer that any potential 52 week outcome for DMS is

reviewed as part of the APESAA process into 2015. However, a slightly higher return rate is

observed in the DMS program. The following table shows the numbers of people that

achieved a 26 week full or pathway outcome before 30 June 2011 that were then re-

referred (it would be interesting to know how many were commenced) back to employment

services within a six month period.

5 52 Week Outcome Discussion Paper issues 25/09/12 to the DESPFRWG


17

Table 1 Returns to employment services

DMS % ESS %

30 June 11 9 801 28% 9 055 26%

31 December 116 18 550 ? 18 304 ?

Request: That DEEWR provide the DESPFRWG with data regarding the number of people

who achieved a 26 week full or pathway outcome on or before 31 December 2011 who were

then re-referred and/or re-commenced in employment services within six months of exit

and by disability type.

For discussion

To support Option 17, the introduction of an unpaid 52 week outcome measure under the

following conditions:

It is renamed as a Sustainability Indicator or 52 Week Indicator because it is a proxy

measure and does not share the same characteristics as other outcomes (in terms of

guidelines, compliance, finance, verification)

An unpaid proxy measure is trialled in DES-ESS only; as recommended by the senate

report

o ESS is funded to provide long-term support (time-unlimited)

o DMS is not funded to provide long-term support and the recent APESAA

report sketches out 13, 26 and 52 week outcomes for future programs

An unpaid 52 week proxy measure in ESS can build an evidence base

help inform program design for 2015 and 2018 (and interaction with

the NDIS)

The 52 week denominator is populated only by participants the provider has

anchored for employment as this is indicative of sustained employment (see

Attachment A)

It has a meaningful performance weighting (15%)

The OSA remains triggered 52 weeks after job placement (i.e. is not brought forward

to 26 weeks, as this will add to the risk of providing ongoing support and work

against the goal of sustainable outcomes and might exacerbate program churn)

A ‘Job End Date’ field is introduced into the Employment Services System

An immediate review of KPI 2.8 to ensure it is fit for purpose

o See Attachment B for a full explanation as to why KPI 2.8 is reported as not

fit for purpose, and can perversely work against program objectives

6 Includes 26 week outcomes for education as well as employment, based on DES Monthly Report as at 31/12/11

7 As described in 52 Week Outcome Measure paper dated 21/08/12


18

Although OS observes a lower return rate of 20%, it is alarming that

77% of those who return do so within six months – from a service

designed to provide ongoing support for as long as required

Modelling alternate handling of numerator and denominator

Possibly moving OS to KPI3 (Quality) and attributing 15% to a

sustainability indicator (unpaid 52 week outcomes) and increasing

the value of full outcomes by 5% respectively while this review is

undertaken under the strict understanding that OS must be

reintroduced as a performance factor when it is fit for purpose

It is verified as per Option 3 described in the 52 Week Outcome paper dated

25/09/12

o Members are reluctant to go to the expense of verifying an unpaid proxy

outcome

However, basing the numerator on referrals to employment services

can be problematic and would cause many appeals and questions.

For example, some clients might be referred to JSA if their income

drops in a fortnight. Others might have longer reporting cycles. It

also picks up those without reporting requirements that might not

be working but aren’t accessing assistance

Refer issues to DES Operational Working Group

DEEWR provides information on the proxy indicator requirements, including its

intention to measure and the use the data

DEEWR provides clarity on the handling of former DES-ESS participants who are still

working but are re-referred to JSA because they are in receipt of income support


19

Attachment A – Sustainability indicator denominator

Premise: An unpaid 52 week outcome measure in DES-ESS would drive providers to a) make

considered job matches and b) result in more appropriate usage of Ongoing Support

resulting in sustainable outcomes post 26 (and 52) weeks

The objective of introducing a 52 week outcome measure is to provide an additional

and specific incentive to DES providers to place participants into quality long-term

sustainable employment (beyond the 26 week outcome), and to also creative a

disincentive for the possible churning of participants post the 26 week outcome.8

However, the proposed handling of the 52 week denominator appears to value quantity

over quality, when the intended incentive is better quality job matches that result in

enduring employment outcomes. We submit that DEEWR can better meet its stated

objective of introducing a 52 week outcome by reconsidering the manner in which the

denominator is populated.

Recommendation: That the 52 week denominator is only populated by those participants

that providers anchor for employment.

Impact: Providers are rewarded for the sustainability of their individual anchors. Anchors

are made at the providers’ discretion; therefore this measure should assess the quality of

those anchors, using tenure as a proxy measure of quality. The existing 13 and 26 week

measures rewarded quantity; this review is about emphasising the quality of outcomes.

Proposed Denominator – Count of participants who have commenced in DES, been

anchored for employment and duration in assistance (EA and PPS phases) is at least 52

weeks (excluding all periods of suspension) or have achieved a 26 week outcome for

employment at least 26 weeks prior to the end of the performance period (including

participants who moved into Ongoing Support).

8 52 Week Outcome Discussion Paper issued 25 September 2012 to the DESPFRWG

the des performance framework - … · red tape and administration kpi 2.8 and ongoing support...

Documents