enterprise ireland experience on monitoring and evaluation of industry led competence centres

Enterprise Ireland experience on Monitoring and Evaluation of Industry Led Competence Centres

Compera Conference, Ljubljana, Sept 15, 2009.

Martin Hussey. Enterprise Ireland

www.enterprise-ireland.com

Presentation Overview

Brief Introduction to Irish Competence Centres

Model for Industry Led Competence Centres

Previous Model (PAT) experiences

CREST ILCC Best Practice

Plan for Future

Discussion


Enterprise Ireland

EI’s mission is to accelerate the development of world-class Irish companies to achieve strong positions in global markets resulting in increased national and regional prosperity.

Our clients are Irish companies engaged in manufacturing or internationally traded services.

Focus on export sales, research and innovation, productivity, starting up and scaling up, and regional enterprise.

Responsible for Competence Centres Programme, but there are other RTDI agencies involved in Collaborative research funding.


EI and the RTDI Landscape in Ireland

Other RTDI Organisations;-

1) Industrial Development Agency (IDA) Ireland, concerned with FDI, play a significant role in funding collaborative R&D for their clients.

2 ) Science Foundation Ireland (SFI), concerned with scientific research in ICT and Biotechnology and more recently Energy.

3 ) Higher Education Authority (HEA), concerned with building the research infrastructure in the Third Level sector.

Other bodies exist for research in health, agriculture, marine affairs etc


Collaborative Models;- Ireland

HEA Programme for Research in Third Level Institutions (PRTLI), large capital and research grants to build capacity.

SFI Centres for Science Engineering and Technology (CSETs) are Academic led and owned with 15-20% matching Industry funds.

EI Industry Led Networks are 100% state funded research projects to an Industry set agenda.


EI Competence Centres – policy and context

Objective;- to increase business activity and expenditure in RTDI in order to generate economic benefit and

sustainable growth

Intervention;- required to address a market failure in high risk R&D.required to encourage collaboration between business and the research provision community.required to translate the training needs of industry into the research community.

Ireland;- Government Strategy for Science Technology and Innovation document calls for the establishment of Competence Centres in Ireland.


Activity – useful outputs of strategic research

Pure basic research

(Bohr)

Use-inspired basic research (Pasteur)

Not a place you’d want to be…

Pure applied research (Edison)

Considerations of use (market suitability)

Quest for fundamental understanding

Risk of market failure

Yes

No

No Yes

Focus of competence centres Source; Vinnova Analysis 2004.


Industry-led Competence Centres Positioning;- Ireland

SFICSET

COMPETENCECENTRE

Cluster

Industry-led Network

Industry Leadership

Academic Leadership

ScaleDurationImpact

PRTLIInstituteEg Conway

HEA network


Model – Industry Led Competence Centre Centre hosted at one site to give clear responsibility/critical mass

Focussed on collaborative research, following an agenda set maintained and revised by industry

Conducted under the direct supervision of an Industry Board with the power to change the direction of the research in line with market requirements

Clear metrics including scale of collaborative research, technology transfers such as licences, spin outs, staff transfers (both to and from industry), and new companies brought into the consortium

Led by a Technology Leader with strong industry background


Model – Technology Leader A Technology Leader will be an Industrially experienced researcher,

employed by a Host PRO, charged with delivering the Research needs of the Industry group associated with the Competence Centre.

The Technology Leader award will be up to €1M annually, inclusive of overhead, for five years.

There will be a significant breakpoint review in Year 3, which can recommend continuation, closure, reductions or extensions.

Funding will be contingent on satisfactory performance, including review by the Industry Steering Board.

The award is to cover the costs of the Technology Leader, a dedicated research team and associated costs.


Technology Leader – Host – Competence Centre Hosting a Technology leader will require evidence of the following;-

Track record in and clear access to relevant research competence in the Technology Area.

Quick and flexible access to facilities, administrative supports and space.

A clear methodology for working effectively with other relevant Institutions.

Empowerment of an Industry Steering Board in setting, reviewing and amending the Research Agenda and in prioritisation and allocation of resources to projects addressing the Research Agenda.

IP and Technology Transfer methods and policies that are aligned with the Competence Centres programme goals as evidenced by commercialisation of research.

Alignment with Institutional Strategy.


Metrics and Indicators

As Best Practice a mix of top-down and bottom up metrics and

indicators will be agreed and form part of the contract (revised annually)

Metrics; number and scale of industrial collaborative research projectslicences, spin outs, spin instaff transfers (both to and from industry), additional R&D activity within companies (BERD)new companies brought into the consortium.

Indicators; Research QualityInternational EffectManagement QualityTraining outputsSoft Technology Transfers


Previous Model - PATs Multiple University Host locations – virtual networked Centres/units

Cost recovery targets year on year, with self-sustaining status as a target

Multiple part-time Research Directors, some permanent technical staff

Funding front loaded – falls away after time

Industry Advisory Board

Metrics were financial, with top down requirements added later

Used liaison with Academia to develop Technology Focus.


PATs Experiences Multiple University Host locations – attribution of results to specific Centres became

difficult. Competition between locations for business !!

Cost recovery – drove Centres away from research towards training, consultancy and technical services

Part-time Directors did not necessarily engage or prioritize the Centre’s activities over host activities. Permanent staff made Centre closure problematic.

Funding front loaded – concentrated on expenditure on capital at the start and under-funded potential growth towards critical mass over time.


PATs Experiences Industry Advisory Board – was advisory only, this did not empower or engage

industry and they drifted away. Centre’s revert to Host academic driven behaviour.

Metrics were financial – Centres focused on meeting financial target only – sought big and easy projects rather than meeting an overall industrial impact goal. Led to concentration on Framework projects and projects with MNCs.

Academic led Technology Focus did not always match the Industry landscape – the work was of more benefit to no EEA companies in some cases.

Management metrics were not used until much later – top down metrics were applied later – This creates difficulties – metrics should be shaped to mirror the growth and maturity of the Centres.


CREST ILCC - Best Practice - Governance

Governance model must reflect policy context and allow the Centre to develop its performance in line with policy goals.

Leadership is key – chair, director and board must be capable of delivering true collaboration and effective management.

Centre should develop and evolve, renewing the Research Agenda and the operational model over time.

Qualified sub-committees are advised for IP management and Scientific/research excellence.


CREST ILCC - Best Practice – Metrics and Indicators

A mix of ‘top-down’ metrics and indicators reflecting policy goals and ‘bottom-up’ metrics and indicators reflecting a Centre’s own view of it’s success is recommended.

Metrics and Indicators should evolve over time as the Centre develops and grows – expectations of this need to be managed from the outset.

Best Practice selection of metrics and indicators should demonstrate the commitment of the stakeholders to collaboration.

Metrics and indicators should be used in the management of Competence Centres Programmes – support success, correct failure.


CREST ILCC - Best Practice – Financing and Sustainability

The report recommended that prescribing ‘self-financing’ status as an aim for Competence Centres was not advisable. This is to maintain the correct type of research activity, addressing a market failure.

Best Practice indicates that Centre Programmes should be long term but finite. The founding logic for a Centre will have shifted over time.

In order to prevent ‘mission drift’ mechanisms should be set whereby funds are concentrated towards multi-lateral collaborative research projects over the period of operation of a Competence Centre.

IP practice and EU State Aids must be given early consideration in Centre establishment.


CREST ILCC - Best Practice – Training and Mobility

The engagement of industry based researchers in Competence Centre activities was seen as key.

Careful consideration must be given to the employment of student researchers as the Centre’s research activities and operations may not align with the student core curriculum of activity.

Centres are excellent training grounds for researchers and a fertile source of new research talent for companies, and should be used as such.

As best practice, research management issues should be addressed and Centre researchers developed as research managers where appropriate.


Monitoring an Evaluation Plans The level of investment in Competence Centres Programmes means that overall Evaluations of Programmes generally consist of large scale independent reviews, involving in-depth interviews/research.

Evaluations are therefore planned for Year 3 of operation and on Programme Closure.

The mid-point evaluation is designed to offer three answers – wind-down, continue or re-scale. Focus is on the quality of management and activities, strong evidence of impacts and transfers to member companies and plans for the future.

As the Programme Aim is BERD, the final evaluation will look back at this impact on member businesses and the wider sector – independent figures available for this annually (so we have a base-line)


Monitoring an Evaluation Plans - Tools EI have full-time personnel based in the Host universities already, monitoring a range of projects and activities. Each Centre will be assigned one of these personnel as their ‘monitor’

Funding contracts are contingent on the delivery of satisfactory annual technical reports and financial claims.

EI has observer status on the Industry Steering Board of the Centre- this means we are aware of all Board meetings and receive the minutes. Boards to meet 6-10 time per annum.

EI fund directly some of the Technology Transfer Offices – their metrics are around commercialisation and will include Competence Centre metrics.

Funding contracts oblige Hosts to seek consent to exploit IP


Monitoring an Evaluation Plans Number and scale of Collaborative research projects

will be monitored via the Board, and updates provided per meeting. Annual report by Technology Leader also required.

Verification of the project activity can be assessed by the Agencies using the company single point of contact.

MIS systems have been enhanced to merge the investment in R&D data from the Higher Education side to the direct company grant side.

Evaluations will take sample case studies to verify validity of overall data reported.

Audit function on a 5-10% sample


Monitoring an Evaluation Plans Licences


Verification of the licence activity can be assessed by the Agencies using the Exploitation Consent Clause. Committee meets monthly.

Hosts must offer IP first to contributors, then to member and then wider – at market rates. Independent assessment for rates available.




Monitoring an Evaluation Plans Start-ups and Spin-ins


Verification of the start-up activity can be assessed by the Agencies using the Technology Transfer Offices. Committee meets monthly.

Attribution of the ‘cause’ of the start-up to be assessed as soon as practical. Spin-in is not an issue.




Monitoring an Evaluation Plans Staff Transfers


Formal transfer can be verified via Host personnel offices, and registration of transfers in where on payroll. Less formally, timesheeting is a requirement.

As Physical Centres – a register of attendance can be employed.




Monitoring an Evaluation Plans BERD

will be monitored via an independent Agency. (Annually)

New Members

Company membership fees (at a nominal rate) are a requirement of the programme.

Board to maintain a register of members.


Indicators

Research Quality – the Board and the Hosts are to agree to a framework of research quality metrics for published work.

International Effect to be demonstrated by participation in EU Framework programme. This is managed via EI in any case.

Management Quality – on Breakpoint Evaluation

Training outputs – measure of effort, awards etc – captured on Evaluations.

Soft technology transfer – not decided, ‘certificate’ scheme in mind.


Industry-led Competence Centres

THANK YOU FOR LISTENING

DISCUSSION

enterprise ireland experience on monitoring and evaluation of industry led competence centres

Documents

collaborative research

research community

scientific research

research grants

research infrastructure

enterprise ireland experience

regional enterprise

enterprise irelandwww