Page 1 of 6

Return on Investment (ROI) Project

Summary Report 26th October 2012

Authors: Dr Alex Collie, Chief Research Officer

Ms Pauline Zardo, Research Translation Officer Dr Sue Gargett, Health Economist

Ms Jane Monk, Consultant

For further information regarding this report please contact: Dr Alex Collie

Chief Research Officer Institute for Safety Compensation and Recovery Research

Level 11, 499 St Kilda Road, Melbourne 3004 VIC, Australia Email: alex.collie@monash.edu

Telephone: +61 (0) 3 9097 0613

Page 2 of 6

ACKNOWLEDGEMENTS Collation of a body of work such as this is only possible with the active involvement of a large number of people. The authors would like to acknowledge the substantial number of individuals within ISCRRs three partner organisations who have contributed to this report. Names and position titles of individuals who have contributed information to the case studies are included in the appendices. In addition we would like to acknowledge Ms Khic Huoy-Prang from ISCRR who assisted with data analysis for some case studies, and Mss May Cheng and Dora Qian from Corporate Finance, Monash who assisted with financial statements data. Of particular note we would like to thank the members of the ISCRR Return on Investment Committee: Mr Chris Latham (ISCRR) and Professor Niki Ellis (ISCRR) for providing leadership

throughout the ROI project including chairing the committee meetings and ensuring the project met the diverse needs of ISCRRs partners.

Dr Duncan Buckeridge (Monash) for critical input into the development of the Impact Assessment Framework and methodology for assessing quantitative impact throughout the project.

Mr David Gifford (TAC) and Ms Julie Evans (WorkSafe Victoria) for substantial

contribution to methods development and for overseeing the quantative evaluation on a number of case studies.

Mr Tehn Chin (WorkSafe Victoria) and Mr David Attwood (TAC) for contributions to case

studies including undertaking a number of quantative analysis and regular engagement and interaction with the project team.

Mr Tony Tenaglia (ISCRR) for contributions to individual case studies and strategic input

throughout the project.

Ms Gillian Syres (ISCRR) for early leadership of the ROI project including contributions to developing the Impact Assessment Framework.

The project would not have been possible without the substantial contributions of the ROI group.

Page 3 of 6

EXECUTIVE SUMMARY A primary objective of the Institute for Safety Compensation and Recovery Research (ISCRR) is to conduct excellent health and safety, compensation and recovery research that positively impacts the performance of WorkSafe Victoria and the Transport Accident Commission. Assessment of its impact, including return on investment is one of ISCRRs key performance indicators. In late 2011, the ISCRR Board initiatied a Return on Investment (ROI) project to assess the impact of ISCRRs research on WorkSafe and the TAC, and to establish a method for assessing the impact of ISCRRs research that can be embedded in ongoing operations of the institute. The ROI project has three components, including:

1. A series of nine detailed case studies of ISCRR research projects to assess in depth the qualitative and quantitative impact of the projects on WorkSafe, the TAC and their stakeholders.

2. A content analysis of information captured in ISCRRs project management system to (a) assess the extent of adoption of ISCRR research by WorkSafe, the TAC and their stakeholders and (b) identify factors that lead to adoption of ISCRR research.

3. A content analysis of ISCRR project level information and the findings of the case studies to determine the number of projects for which impact could be demonstrated as well as the types of impact.

The total financial investment in the nine case studies to date is $725,000. The financial return on investment has included a $1.5 million actuarial release from the Noise Induced Hearing Loss project (a five fold ROI), a $1.8 million cost avoided from the Implantable Pain Therapies evidence review, and costs avoided of up to $6.8 million for the Body Weight Supported Treadmill Training evidence review. The financial impact of three TAC focussed projects is yet to be determined. These estimates will be known in coming months as the impact of the research on claims costs at TAC become clearer. In addition, the non-financial value of the projects has been substantial, and includes positive changes in the culture of evidence-based decision making at both WorkSafe and the TAC, increasing the understanding within the TAC of the impact of claims management processes on TAC clients, improvements in research capability amongst leading injury research group at Monash, enabling the implementation of Recovery model at the TAC, establishing a basis for important health policy positions at WorkSafe and the TAC, and enabling enhanced interactions between rehabilitation coordinators and TAC clients. The research has also enabled the continuance of an important workplace safety debate in the community (i.e., quad bikes safety devices). The impact of the case studies is summarised in the table below. Analysis of data captured in ISCRRs project management system identified evidence of use of ISCRRs research in 27 of 30 individual projects that had produced output by end August 2012. In 17 of these projects qualitative information demonstrating the impact of the projects on WorkSafe, the TAC or their stakeholders has been collected. Content analysis identified five major types of impact, being financial impacts (such as reductions in actuarial liabilities and costs savings), claims processing and claims decision making impacts, impacts on employers and the broader community. In addition, one ISCRR project has demonstrated an impact on the health and wellbeing of compensable clients with severe injury.

Page 4 of 6

Table: Summary of ROI from nine case studies CASE STUDY INVESTMENT RETURN Noise Induced Hearing Loss

$312,000 $1.5 million actuarial release Allowed WorkSafe to focus its strategic response to

growing hearing loss costs

Quad bike Safety Devices $10,500 Re-ignited public debate around important workplace safety issue

Contributed to a 'case for change' in safety standards Contributed to community behavioural change

Client Perceptions of Recovery

$37,700 Enhanced understanding of TAC role in improving client experience

Provided information for the education of claims officers

Provided opportunity for key researcher to engage with TAC

Contributed to capacity building in research team at Monash

Implantable Pain Therapies

$73,400 $1.8 million in costs avoided consisting of $0.65m in costs saved over 3 years to June 2012 and $1.16 million in future costs avoided.

Contributed to change in culture towards evidence based decision making

Helped WorkSafe to contain a potential "cost-blowout" Leant credibility of WorkSafe policy of reviewing IPT

requests Body Weight Supported Treadmill Training

$65,600 Contributed to a shift towards evidence-based decision making in TAC.

Provided a foundation for an important health services policy position.

$17k to $342k in current costs avoided, depending on scenario modelled.

Potential liability saving of up to $6.8 million in costs avoided.

Musculoskeletal Disorders Toolkit

$71,600 In one participating organisation, involvement in the research led to: increased knowledge and skills re work-related MSK

disorders, increased communication on OHS in the workplace, increased involvement of staff in OHS issues, and the introduction of initiatives to enhance skill

development. Pelvic Ring Fractures $36,000 Improved physical and mental health outcomes for

severely injured TAC clients where there was a surgical intervention.

Financial impact yet to be determined.

Page 5 of 6

Evaluation of the Client Conversational Tool

$106,300 Provides TAC Rehabilitation Coordinators with a greater insight into client needs.

The CCT-revised is more usable, more likely to be administered in specified timeframes and more likely to be completed by rehabilitation coordinators than the CCT.

Financial impact yet to be determined. Return to Work Predictive Model

$10,000 plus $500 in ongoing operational costs p.a.

Enabled Recovery model implementation Led to earlier interactions with TAC clients Facilitated enhanced client / rehabilitation coordinator

relationships. Financial impact yet to be determined.

Content analyses identified eight factors that influence the adoption of ISCRRs research including engagement and interactions, alignment with partner strategic priorities, timeliness of research, ease of implementation, partner organisations structure and processes, risk and issue prioritisation, the presence (or absence) of an internal champion for the research, and the credibility of the research method and source. This analysis provide substantial value to ISCRR, as it informs ongoing development of our project development and translation system to ensure that research findings are more likely to be adopted. This analysis identified a substantial return on investment from ISCRR projects to date. While there is some inherent uncertainty in the financial estimates, the financial ROI calculated for three of the case studies demonstrates a return on investment that is many times the investment in the research. Other projects have demonstrated the substantial non-financial value for WorkSafe and the TAC from ISCRR research. The analysis has been undertaken on a small proportion of completed projects from the ISCRR research portfolio. As ISCRRs research program matures the financial and non-financial benefits to WorkSafe and the TAC may grow.

Page 6 of 6

REFERENCES Clay MA, Donovan C, Butler L, Oldenburg BF. The returns from cardiovascular research: the impact of the National Heart Foundation of Australias investment. Med J Aust 2006; 185: 209-212.

Collie A. Gains in neurotrauma research activity and output associated with a Victorian state government funding program. Medical Journal of Australia 2010; 712-714. Dobrow, M.J., Goel, V., Upshur R.E.G. Evidence Based Health Policy: Context and Utilisation. Social Science and Medicine 2004; 58: 207-217. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. Diffusions of Innovations in Service Organisations: Systematic Review and Recommendations. Milbank Quarterly 2004; 82(4): 581-629.

Insight Economics. Economic impact study of the CRC programme. Canberra: Insight Economics, 2006. http://www.universitiesaustralia.edu.au/documents/news/media_releases/2006/CRC_Economic_Impact_Study-Oct06.pdf (accessed Apr 2010)

Kingwell BA, Anderson GP, Duckett SJ, et al. Evaluation of NHMRC funded research completed in 1992, 1997 and 2003: gains in knowledge, health and wealth. Med J Aust 2006; 184: 282-286.

Lavis, J. How Can We Support the Use of Systematic Reviews in Policymaking? Plos Medicine 2009; 6(11).

Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. Journal of Health Services Research and Policy 2005: 10(1); 21-34.

Mitton, C., Adair, C.E., McKenzie, E., Patten, S.B., Waye Perry, B. Knowledge Transfer and Exchange: Review and Synthesis of the Literature. Milbank Quarterly 2007; 85(4) 729-768.

Orton L, Lloyd-Williams F, Taylor-Robinson D, O'Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One 2011; 6(7).

Schapper CC, Dwyer T, Tregear GW, Aitken M, Clay MA. Research performance evaluation: the experience of an independent medical research institute. Aust Health Rev. 2012 May;36(2):218-23. Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, Jones CA, Klassen TP, Kovacs Burns K, Newton AS, Thompson D, and Dryden DM. Systematic review of knowledge translation strategies in the allied health professions. Implementation Science. 2012; 7(1):70.

Wells R, Whitworth JA. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure? Aust New Zealand Health Policy 2007; 4: 14-17. http:/ / www.ncbi.nlm.nih.gov/pmc/articles/PMC1929109/ (accessed Apr 2010). Zardo P, Collie A. Use of evidence in public health policy: An injury compensation policy content analysis. Accident Analysis and Prevention (submitted).