nes measuring imapct jan 2012

xxxxxx

Measuring Economic Impact- Overview -

January 2012

Rupert Greenhalgh

Director: NES Consulting

xxxxxx

About us…

Economic policy & strategy

Local economic & skills assessment

Project evaluation & cost benefit analysis

Consultancy

ProBono Economics

Think Pieces & Thought Leadership

New Economic Strategies

xxxxxxPerformance framework

Two key components:

A ‘top-down’ component:

monitors the changing state of conditions in the functional economic area / administrative area

requires the use of ‘Headline Indicators’ & ‘Condition Indicators’

A ‘bottom-up’ component:

enables the Strategy’s activities, outputs & outcomes/impacts to be assessed

uses appropriate ‘Response Indicators’

xxxxxxHeadline indicators (1)

Should be embedded within the strategic process:

Plan > Do > Act > Check

Clear & robust insight into strategy’s efficiency & effectiveness

Outputs & outcomes aligned to overall achievements

Use & development of the framework will be iterative

Devised to yield additional information

What actions are more effective & efficient than others?


Focused on the added value of the Economic Strategy

Reflect the totality of value added rather than individual priorities

Provide an outward facing assessment of progress

Enable comparison of performance against other areas

Widely understood by Gov & business

Provide a common evidence base to share local economic performance with all key stakeholders


Delivering growth & prosperity:

1.GVA per capita (as a measure of resident prosperity)

2.GVA per worker (as a measure of employee productivity)

3.Total jobs (employee jobs located within workplaces)

4.Growth in private sector employment (representing & additional measure of re-balancing)

5.Business density (measured by the total number of businesses per 1000 population)

6.Other measures? Discuss

xxxxxxExample 1: Setting Targets and Monitoring

Indicators

xxxxxxCondition indicators (1)

Business support

Condition Indicator Data / definitions

GVA per employment in transformational sectors

£GVA in constant prices

Stock of VAT registrations per 10,000 working age residents

VAT registered enterprises

% of small businesses in area showing GVA growth

VAT registered enterprises

Workforce median weekly wagesAll employees working within region


Condition Indicator Data / definitionsTotal GVA from Advanced Manufacturing

£GVA in constant prices

Proportion of enterprise turnover due to new or significantly improved products

Local enterprise survey

Perception of region as a place of innovation & business opportunity

Local, national & international perceptions from commissioned surveys

Number of patents granted to enterprises in the region

Patents granted (not applications)

Innovation



New VAT registrations per 10,000 working age residents

Working age population aged 16-64 yrs

Stock of VAT registrations - cohort surviving 36 months

VAT registered as % of original cohort of start-ups 36 months previous

% of small businesses in an area showing employment growth

% of total SME’s growing in last 12 months

% of 16 to 24 year olds not in employment, education or training (NEET)

National definition for NEET

Entrepreneurship


Condition Indicator Data / definitionsNumber of new graduate level jobs created through inward investment each year

Assisted by regional stakeholders in last 12 months

Number of Foreign Direct Investment (FDI) projects in region each year

New FDI projects locating within year

Annual perception survey with key national (public & private sector) stakeholders

Commissioned survey supported by range of qualitative measures

Number of trade enquiries through inward investment agencies

Monitoring information of key partners

Number of targeted communications published aimed at key stakeholders

Monitoring information of key partners

Inward investment & place marketing



% of working age residents qualified to Level 2+


% of working age residents qualified to Level 4+


% of working age employees in LCR qualified to Level 4+


% of 15 year olds getting 5 or more GCSE/GNVQs at grades A*-C

Pupils in public & private education

% of working age residents undertaking job related training

Working age 16 to 64 years within last 13 weeks of survey

Labour market & workforce skills


Major projects


Amount of accommodation floor-space brought forward (sq m)


Value of Local Economic Multipliers (LM3) secured through major developments


xxxxxx`Strategic Added Value (SAV)

leadership & catalyst: demonstrating the benefits / communicating effectively development needs to stakeholders

strategic influence: affecting the behaviour & allocation of funds &/or activities by stakeholders

leverage: securing funds to contribute to wider objectives

collaboration: improving exchange & synergy between stakeholders, encouraging co-ordinated stakeholder activity

engagement: openness & involving stakeholders in the design & delivery of activities

empowerment: empowering responsibility to people & organisations best placed to deliver effective & efficient outcomes

xxxxxx

Consultation on the condition indicators will be important to highlight that valid issues are being measured. These should be reviewed periodically to ensure they represent priorities

The framework should be used as a tool with local partners to negotiate and agree the outputs required to successfully contribute to the outcomes required by the strategy

Time must be allocated upfront to support stakeholder’s understanding of what is required to support the frameworks implementation

Performance monitoring will/should already be taking place. It is important that collaborative work between organisations takes place at the start to reduce costly duplication

Robust monitoring and evaluation processes must underpin the Economic Strategy. Systems should be put in place at the start of strategic planning to capture the outcomes of activity

Some other considerations: Implementation

xxxxxx

Other approaches:

Evaluation

Social accounting and audit

Logic models

Theory of change

Balanced score card (for local enterprise)

Local Multipliers

Social Return on Investment

CBA…

And finally…

xxxxxx

Linking project to impacts & addressing ‘gross to net’

xxxxxx

What is evaluation:

“a process of measuring or assessing success”

“learning lessons for the future”

Looks critically at what has been achieved through a project, programme or other intervention

Other similar activities:

monitoring

performance management

Valuing costs and benefits (1)

xxxxxx

Why is evaluation important:

To understand impact & assess whether objectives have been met

To identify learning, informing strategy, resource allocation, delivery

To demonstrate accountability, value for money, & contribution to wider strategic objectives

To develop evidence for funding bids, promotions, target resources

Provides a rigorous framework - or ‘Logic Model’ that links project aims, inputs, outputs, outcomes, & lessons learnt


xxxxxxContextual conditions

Policy conditions

Rationale Inputs Activities

Intended Outcomes

Outputs

Intended Impacts

Programme Objectives

Project:

xxxxxx

1. Deadweight – questions about what would have happened in any case?

consider baseline trends, map impacts of activities, interviews with partners

2. Leakage – proportion of outputs benefiting people from outside target

monitoring data (e.g. jobs for local residents) or beneficiary (e.g. JSA)

3. Displacement – degree to which the project reduces activity elsewhere

qualitative questioning & changes in secondary data

4. Multiplier effects – the positive knock-on effects on the wider economy

specialist studies/methodologies (e.g. LM3), or use of benchmarks


xxxxxx

HMT Green Book (HMG Appraisal and evaluation): http://www.hm-treasury.gov.uk/data_greenbook_index.htm

Magenta Book (HMG Policy evaluation): http://www.hm-treasury.gov.uk/data_magentabook_index.htm

White Book (DCMS Creative Industries): http://www.compass.unibo.it/Materiali/Documenti/2007/Documenti_Mazzanti_whitebookDCMS%20UK.pdf

Historic Environment (Appraisal and evaluation):http://www.hlf.org.uk/aboutus/howwework/Documents/Valuingthehistoricenvironment_eftecFinal_Report.pdf

Assessing the Impacts of Spatial Interventions (3Rs):http://www.communities.gov.uk/documents/regeneration/pdf/156906.pdf

Orange Book (Appraisal and risk management):http://www.hm-treasury.gov.uk/media/A/5/managingrisks_appraisal220705.pdf

SROI Network and supporting training materials:http://www.sroi-uk.org/publications-uk/cat_view/29-the-sroi-guide-2009

Resources

xxxxxx

Friendly, free advice/information

Collaboration – research projects

NES Consulting

info@nesconsultingcouk

Contact us at New Economic Strategies

nes measuring imapct jan 2012

Business

working age population

working age employees

monitoring indicators

local economic performance

months vat

economic strategy

total number of businesses

employment growth