Measuring Impacts: Building Support for

Economic Development

Kenneth E. Poole, Ph.D.President/CEO

2010 Best Practices ConferenceMAEDCMay 6, 2010

About the Council for Community and Economic Research

• National membership group, created 1961

• Data products– ACCRA Cost of Living Index– Business Finance and

Incentives Database– Economic Development

Organizations Survey

• Training resource– Regional economic analysis

research methods training– LMI Training Institute

• Technical assistance– Statewide survey of green

jobs– Local community & economic

research support for projects

• Advocacy for local community and economic data

2

Overview

• Why Measure? Why Evaluate?

• Ways to Measure Performance and the Logic Model

• Case Examples

• Tools for Measuring Impact

• A Few Key Issues & Some Words of Advice

3

THREE KEY APPROACHES

Evaluate, Benchmark, or Project: What’s the Difference and Why Does It Matter?

4

Why Bother?

• Key Thesis:– Performance builds credibility for and

confidence in your economic development efforts

• Options for building credibility…– Money– Friends– Data – Our focus today

5

What is “Performance”?

• How has the economy or the fiscal situation changed over time?

• How well did you achieve public goals with your economic development investments?

• How can you improve your economic development programs?

• What can you predict about future impacts resulting from your investment?

• How does your community compare with others?

6

Key Distinctions

• Evaluation – Examining investments made to assess

impact• Benchmarking

– Monitoring socioeconomic measures to identify challenges or opportunities

• Projection– Using known or speculated impacts to

predicting the most likely future result

7

Key Differences Between Evaluation & Benchmarking

Evaluation• A process, not a single

event• Compares “What is” with

“What would have been” or “What should be”

• Search for answers from the program itself

• Applied research related to ongoing investments and/or programs

• Practical solutions, not to be sit on the shelf

• Data based on program information

• A snapshot, often repeated periodically

• Compares across places• Often, exploratory, seeking

to identify problems or successes

• Focuses on identifying resource gaps to address problems

• Data based on independent statistics

8

Benchmarking

Making Your Case: Key Issues in Selecting the Right Approach

Projecting Investment Impacts• Do the assumptions pass

the smell test?• Are you accounting for

expected dislocation?

• Have you selected the right measures to compare?

• Does your planned investments contribute significantly in affecting those metrics?

Measuring Program Impacts

• How much did investment really influence (or “cause”) business behaviors?

• How strong is the data available?

9

Benchmarking Progress

Challenges in Conducting Evaluations

• Concern about how evaluation research will be used

• Lack of credibility given to internal evaluations

• Lack of expertise among program managers in:

– Evaluation techniques

– Data collection process

• Lack of resources to conduct evaluation

– Concern that diverted resources take away from program success

May 7, 2010 10

Creating an Evaluation Logic Model

11

Inputs /Resources

• Funding Provided

• Staffing

• Facilities Available

• Partners Engaged

Program Outputs

Program measures

tied to specific

Program Objectives

Combined report of program

impacts (or “predicted” program impacts)

Activities

Milestones and

Deliverables

Generated from

tactics or tasks

designed to achieveparticularresults

Socio economic Outcomes

Benchmarkindicators of progress towards a Strategic

Vision.

Might not be directly tied

to any program,

but should betied toVision

•Guiding•Detecting•Formulating

Linked to Strategic Objectives

Types of Measures

What Do We Measure?

• Jobs created or retained

• Private investment leveraged

• Total public investment

• Tax revenues generated

12

Need for More Compelling : Lessons from Benchmarking

13

• Knowledge Jobs (“Quality Jobs”)– Information tech jobs– Mgt, prof, & tech jobs– Workforce education– Immigration of knowledge workers– US migration of knowledge workers– Mfg value-added– High-wage traded services

• Globalization (“Quality Markets”)– Export focus of mfg and services– Foreign direct investment

• Economic Dynamism (“Vitality”)– ‘Gazelle’ jobs– Job churning– Fastest-growing firms– Initial public offerings– Entrepreneurial activity– Inventor patents

• The Digital Economy (“New Infrastructure”)

– Online population– Internet domain names– Technology in schools– E-government– Online agriculture– Broadband telecommunications– Health information tech

• Innovation Capacity (“Emerging Opportunities”)

– High-tech jobs– Scientists and engineers– Patents– Industry investment in R&D– Non-industry investment in R&D– Movement to a Green economy– Venture Capital

Source: 2008 State New Economy Index Indicators

Challenges with Measuring

• Sporadic and inconsistent data collection

• Rarely validated by a third party

• Data not always relevant or used

• Limited priority given to data collection (or analysis)

• Concern about negatively impacting business relationship 14

EXAMPLES TO HELP YOU FRAME YOUR THINKING

Case Studies: Benchmarking & Projections and Evaluation

May 7, 2010 15

Indicators & Benchmarks

Census Bureau

Local Employment Dynamics (LED)

Community Economic Development Hot Reports

May 7, 2010 16

Benchmarking Over Time

Maine Economic Growth Council

Measures of Growth

May 7, 2010 17

Projections / Impacts

• Example:• 200 new distribution jobs in Milwaukee /

Madison– What is the likely job impacts – 468 includes:

• 256 in the transportation and warehousing• 32 in retail trade• 31 in health care

– What is the likely regional economic impact -- $25.8 million

• Construction of facility is considered separately

• Dislocation is not often considered at all

May 7, 2010 18

Model for Evaluation: Portfolio

• Maine Comprehensive Economic Development Evaluation System

19

2 parts – innovation and traditional econ dev programs1. Innovation (i.e., technology

programs) completed for past 9 years

2. Traditional business development -- first completed in 2009

Benchmark outcome measures tied to:o Comparator stateso Annual Maine Economic

Growth Council benchmark report

Third-party survey of business and community clients

Funded by an annual assessment on every program being evaluated

Impact Scenario: Sample

Ownership ScenarioAnnual Impact on

Value AddedAnnual County-WideEmployment Impact

Community Wind (5%opportunity cost of capital)

$1,259,188 14.5

Community Wind (8%opportunity cost of capital)

$639,739 8.2

Corporate Wind $249,388 4.3

20

Economic Impact of Local vs. Corporate Ownership of Wind Energy Production in Big Stone County, MN (operations phase)

Source: Arne Kildegaard and Josephine Myers-Kuykindall, University of Minnesota, Morris, “Community vs. Corporate Wind: Does It Matter Who Develops the Wind in Big Stone County, MN?, Sept. 2006

Model for Evaluation: Program

NIST Manufacturing Extension Partnership

21

• Program Objectives– Reduce business costs– Expand business revenues– Also tracks…

• Jobs created and retained• Measures program efficiency (e.g.,

dollars invested per client assisted)

• Evaluation includes:– 3rd party business survey of

assistance recipients– Minimally Acceptable Impact

Measures (MAIM) index– Panel reviews – colleagues

assessing other centers– Case studies of successful

venturesSample MAIM Scores

THE NITTY GRITTY DETAILSTools and Techniques for Evaluating or Projecting Impacts

22

Using Economic and Fiscal Impact Analysis as an Evaluation Method

Economic Impacts• Jobs created/retained• Payroll generated

(average wages)• Private investment

leveraged• Businesses or

entrepreneurs assisted• Jobs per dollar invested

Fiscal Impacts• Tax revenues

generated– Examples: Sales taxes,

property taxes, income taxes, etc.

• Government costs occurred– Direct or indirect one-time

or on-going expenses

23

24

Understanding Importance of Economic Interdependencies

Inter-industry linkages (purchasers )

Direct Consumption

Inter- industry linkages (sellers)

Business-to-business purchases (indirect impact)Examples:

Autos buying plastic partsMedical buying instruments

Retail sales

Sales to out-of-area customers

Purchases of Labor & Value Added

WagesInterestProfits

25

Economic Multipliers

• Recognizes inter-industry linkages (“pebble in a pond”)

• Quantifies “recycling” of dollars within and “leakage” of dollars from the local economy

• Forces the analyst to consider BENEFITS and COSTS associated with “indirect” impacts

• Provides “simple” tool for estimating overall economic impacts (current or future)

26

Calculating a Multiplier

• Differs depending on “unit of measure”– Employment

– Business Revenues

– Personal Income (e.g., wages, transfer payments)

• Multiplier = (Direct + Indirect + Induced)

Direct

• Includes the direct effects

27

Rules of Thumbs

• For metros-employment multipliers of greater than 3.0 are VERY unusual

– Manufacturing: • Typical range = 2 to 2.5

– Non-manufacturing: • Typical range = 1.3 to 1.8

28

Common Errors in Using Multipliers

Made up or misunderstood….Regional preferences differDefining the industry (“What’s the multiplier for manufacturing?”)Defining the study area (The “region” matters….)Changes in technology

29

Common User Errors in Conducting Impact Analyses

• Emphasizing benefits without recognizing negative consequences – Displacement issues

(What were residents doing before there was a baseball stadium?)

– Multiplier effects (What are the public services required for immigrants to the community?)

• Emphasizing costs without recognizing positive consequences– Displacement issues

(What are the true costs if we are not adding a lot of new economic activity?)

– Multiplier effects (What are the revenues being generated from new business activity, especially increased retailing?)

• Timing of Impacts– Investment Today, Impacts Tomorrow

30

Fiscal Impact Analysis – Summary of Costs

• Additional public services required for project success

• Additional public services required to respond to added population

• Expanded infrastructure needed to handle new business activity

• Cost of incentives –– Direct, indirect, and tax benefits funded through public

investment– Allocation of incentive – the “public” benefit vs. the

“private” benefit from the investment

31

The Key Issues in Conducting Fiscal Impacts

• Generating good estimates of economic impacts

• Defining relevant costs• Estimating costs (average vs. marginal costs)• Balancing the presentation of benefits and

costs• Equity issues (who is impacted is important)• Difficulty in generalizations across

jurisdictions

32

Software: Economic Impact Models

• U.S. Bureau of Economic Analysis (RIMS II)– http://www.bea.gov/bea/regional/rims/

• Minnesota Implan Group, Inc. (IMPLAN)– www.implan.com

• Regional Economic Models Inc. (REMI)– www.remi.com

• Local university analysts – proprietary models

33

Software: Fiscal Impact Models

Federal Reserve Board “Fiscal Impact Tool”http://www.federalreserve.gov/community.htm

Local university analysts – proprietary models

Budget, planning, or economic development offices combine commercial models with their own tweaks

e.g., Maryland Resource Allocation Model

Other proprietary models

Modeling or Tools Used To Measure Impacts

• Frequently used economic modeling tools– RIMS II – U.S. Bureau of Economic Analysis– Implan – Minnesota Implan Group, Inc.– Analyst – Economic Modeling Specialists, Inc.

(EMSI)– Policy Insight – Regional Economic Models,

Inc. (REMI)

34

RIMS II – U.S. Bureau of Economic Analysis

35

• Strengths–Models are built using

unsuppressed gov’t data–Lots of Industry detail

(473)–The only model with

general public name recognition (BEA)

• Weaknesses–Relatively simple technique

used to develop multipliers–No software; user interface

for a series of spreadsheets–No forecasting or historical

data–No access to underlying

data

Implan: Minnesota Implan Group

36

• Strengths–Lots of industry detail;

500+ industries–Detailed agricultural

sectors –Flexible model

configuration–Multi-region capabilities

• Weaknesses–Somewhat simple

modeling techniques–No time series or

forecasting capabilities

Analyst : Economic Modeling Specialists, Inc.

37

• Strengths–Web subscription–More than 700 industries; 800

occupations–Can adjust estimated exports

(relevant for small areas)–Zip level data allows easy-to-

build sub-county models– Includes basic cluster analysis

(using pre-defined clusters from other research)

–Part of a suite of products “Forecaster” and “Career Pathways”

• Weaknesses–Static model–Relatively new to the economic

development market (released in 2005)

–Not well known and greatly tested for this purpose

–Initially created as a workforce training product

–Academic articles available, but not comparative

Policy Insight: Regional Economic Models, Inc. (REMI)

38

• Strengths–Attempts to model a

great deal of economic behavior

–Can model multiple regions

–Includes a 50-year forecast (for transportation planning)

–Most well known of dynamic modeling system

• Weaknesses–Difficult to use correctly–Very limited industry detail–Most expensive

Which Tool Should You Use?

39

For the “best” multipliers – all provide relatively similar results – RIMS II typically largestFor simple impact studies – all can be useful, cost may be a factorFor lowest cost – RIMS II, followed by IMPLANFor brand recognition – RIMS II, REMI, and IMPLANFor forecasting – REMIFor geographic detail – IMPLAN and EMSI (zip code level)For the “most up-to-date” data – EMSI (quarterly)Web access – EMSI, REMIFor phased projects or policy analysis -- REMIFor cluster analysis -- EMSIFor fiscal impacts – crude tools in several of the models

Fiscal Impact Tool

• Federal Reserve Fiscal Impact Tool (FIT) Free

• Useful to evaluate fiscal impacts of local economic development projects– This tool was developed to enable local economic and

community developers to learn more about the local tax impacts of economic development projects.

– The Excel workbook provides defaults and simple assumptions enabling the user to input specifications of a project and quickly ascertain what sales and property (and other) tax gains will accrue to an area.

40

Fiscal Impact Tool (cont.)

• In addition to performing an automated tax revenue assessment, this tool can be used for many data purposes.

• The workbook provides:– Census of Retail Trade data on the number of retail

establishments, amount of retail sales, and the retail sales per capita and dollar of income of counties and cities,

– Historical city and county population estimates, and – County per capita income figures.

• The tool is available at http://www.federalreserve.gov/forms/fiscalimpactrequest.cfm• Not being fully supported by the Fed in the future

41

A FEW TIPS AND OTHER WORDS OF ADVICE

Lessons Learned From Experience

42

What ‘s Next – How to Tell the Story of Econ. Dev. Impacts

• Communication Strategies– Written vs. oral

• Know Your Audience – Policy makers, legislators, media, general public, program managers, clients

• K.I.S.S. (Keep It Short and Simple)• Importance of an Independent Evaluation

43

Why We Often Can’t Determine Economic Development Success

44

1. Lack of a clear vision or goals– What’s the logic behind

the intervention?– What’s the best

measure - jobs?

2. Policies not aligned to state’s vision or goals

3. Lack of program resources– Can’t do enough of

anything to have an impact OR

– Trying to do everything

4. Policy and program design excludes evaluation component

5. Lack of resources for evaluation

6. Success expected too quickly

Recommendations for Moving Forward

45

• Demand a “vision”; toward which one can benchmark

• Require measures; but don’t get hung up on “jobs”

• Measure public return on public investment (ROI)

• Reward actions rather than promises

• Examine the entire investment portfolio; don’t get hung up on individual programs

• Invest in the evaluation system, including data management and third-party assessments

Some Words of Advice

46

• Spin-off impacts– Real, but often

overstated– Question earnestly any

“multiplier impacts” over 2 times the direct impact

• Measuring “good jobs”– 120% average wage

rule of thumb– Certain occupations are

more likely to offer benefits

• Legislators or councilors who play “gotcha” won’t get cooperation

• Use qualitative as well as quantitative measures– Case studies– Capacity building

• Evaluate for continuous improvement

Thank You!

Ken Poole– www.c2er.org– 703-522-4980, ext. 1016– kpoole@c2er.org 47