wojan - subject base innovation research 2014 ers rural innovation survey

The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.

Putting the Subject Back into Subject-Based Innovation Research: Latent Class Analysis in the 2014 ERS Rural Establishment Innovation Survey

Timothy R. WojanEconomic Research Service/USDA

Paper presented at OECD Blue Sky IIIGhent, Belgium

19-21 September, 2016


Outline of Talk• Strong priors that rural innovation is rare and

largely inconsequential • Challenge to conventional wisdom requires

credible measure of substantive innovation• Assume experiences of substantive innovators

unique and can be elicited with simple questions

• Do identified substantive innovators satisfy tests of internal and external validity?

• Feasibility and assessment of “rural innovation policy” requires credible measure of substantive rural innovators


CIS findings contradict but do not overturn conventional wisdom

• NBER, Brookings, World Bank either wholly disregard or disqualify rural in regional studies of innovation

• CIS findings on rural innovation based on response to single ambiguous question

• North and Smallbone (2000): 49% of rural UK mftrs regarded selves as “innovative” based on CIS response but industry experts rated only 24% as “highly innovative”


2014 ERS Rural Establishment Innovation Survey

• First nationally representative self-reported innovation survey for Rural America

• Oversampled rural establishments but allocated a quarter of the sample to urban establishments for comparison

• Sample size 11,000 for all establishments with 5 or more employees in nonfarm, tradable sectors

• Sought more efficient way of IDing substantive innovators


Assume that struggling with innovation alters responses to key

questions• EU CIS core questions in combination with

other observable characteristics –New or significantly improved goods, services,

processes, logistics, marketing methods.–Are innovation investments capital constrained?–Acknowledge failed innovation initiatives?–Possess intellectual property worth protecting?–Does data drive decision-making?


IDing Substantive and Nominal Innovators Using Latent Class Analysis (LCA)

• Assumes that sample drawn from distinct but unobservable subpopulations inferred from the data

• Latent class analysis resolves two main problems of classification in large datasets:– Classification is probabilistic – Can be estimated incorporating complex

sample design with the MPlus statistical package


Latent Class Analysis with Covariates SchematicOutcomes

Outcome Explanatory Vars.

Latent Classes

Covariates Explaining Class Membership

y1 y2 y3 y4

zi … … …. zk

33.09%

36.79%30.12%

xis xks Core Innovation

CovariatesData Driven Decision

Making Covariates

SubstantiveInnovators

Nominal Innovators

Non-Innovators

Source: 2014 Rural Establishment Innovation Survey


0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%Affirmative Responses to Variables Used to

Determine Latent Class Membership

Substantive Innovators Data-Drvien Nominal Innovators Non-Innovators

Core Innovation Ques-tions

Data Driven Decision-Making Questions

% Answering

YES


Whether these subpopulations truly exist is an empirical question

• Initial results will be in cross-section:– Do auxiliary questions provide a sufficient threshold?– Are establishments in more innovation intensive sectors more

likely to respond affirmatively to auxiliary questions? • Linking REIS to the longitudinal business data at

BLS or Census will provide dynamic performance data to compare substantive with nominal innovators

• Broad but shallow survey research supplemented with narrow but deep case study research


Purchase or License Patents Participated in a Patent Application Registered an Industrial Design Registered a Trademark Produce Material Eligible for Copyright

17.38%16.63%

8.35%

30.97% 31.23%

6.02%

2.31%

1.10%

6.15%

8.31%

4.69%

3.02%

0.96%

5.66%

8.11%

Validity wrt Survey Responses: Innovation Related Activities

Substantive Innovators Data-Drvien Nominal Innovators Non-Innovators

% Answering YES


Source: Shackelford 2013 and 2014 Rural Establishment Innovation Survey

Pharm

Instruments

Aerospace

Semicond

Computers

Software

BasicChem

OthChemMedEq

R&Dserv

Auto

MfgNEC

InfoNEC

CompSysDes

ArchSvcs

ProfTechNEC

NMfgNEC0

2

4

6

8

10

12

14

16

18

Rank Order Correlation Between NSF and REIS Innovation Intensive Industries Removing Likely Outlier (NAICS 3342 Communications

Equip.)

NSF Patent Apps REIS

(21)

(43)

(24)

(48)

(9)

(13)

N = 13 but no Metro Substantive innovators

(9)

(135)

(45)

(75)

(112)

N in Parenthe-ses

(2374)

(751)

(195)

(431)

(1932)

(4206)

Rank Order Correlation = 0.433**

NSF

and

REIS

Rank

of

Industry


The central question: Are rural substantive innovators common or

rare?

Substantive Innovators Data Driven Nominal Innovators Non-Innovators

Nonmetro 22.56 38.52 38.92Metro 31.27 32.26 36.47Small Establishments

Nonmetro 18.02 38.29 43.69Metro 26.00 33.18 40.83

Medium Establishments

Nonmetro 28.53 41.12 30.35Metro 41.10 31.96 26.94

Large Establishments Nonmetro 52.14 29.99 17.87

Metro 48.36 22.97 28.67

Hi-tech Manufacturing

Nonmetro 44.04 29.53 26.43Metro 35.56 30.26 34.19

Hi-tech Services Nonmetro 32.71 26.75 40.54

Metro 40.41 24.21 35.38



How Reliable Measures of Rural Innovation Can Aid Rural Policy

• Does rural policy need to address the problems that emerge from innovation-led growth?

• Are market failures that plague sparsely populated areas impeding grassroots innovation?

• How are rural areas best able to ameliorate the disadvantages of distance and kindle the creative spark?


Thank you

Comments? Questions?

[email protected]


Is the innovation measure picking up things that citizens care about?

• Associating substantive innovation with establishment performance such as productivity, exports, employment growth, survivability, etc. must wait for these data to become available

• In the meantime, retrospective employment experience possible based on 2014 county-industry innovativeness estimate and county-industry employment growth in recovery 2009-2014.


Table 9: Regressions of County-Industry Employment Growth, 2009-2014

Variable Parameter Estimate

Standard Error t Value Pr > |t|

Probability Substantive Innovator

82.69 43.02 1.92 0.0546

Share Introducing New Products or Processes

(CIS Equivalent)

-60.61 37.88 -1.60 0.1097

Probability Nominal

Innovator-116.0698 54.081 -2.15 0.0319

Probability Non-

Innovator-14.59 54.01 -0.27 0.7870

Source: 2014 ERS Rural Establishment Innovation Survey and BLS Quarterly Census of Employment and Wages Coefficient estimates for intercept, population, and industry controls not reported


Table 10: Regressions of County-Industry Employment Growth, 2009-2014, Selected

Sectors

Source: 2014 ERS Rural Establishment Innovation Survey and BLS Quarterly Census of Employment and Wages Coefficient estimates for intercept, population, and industry controls not reported

Industrial Sector Variable Parameter Estimate

Standard Error t Value Pr > |t|

Fiber Probability Substantive Innovator

38.64 132.15 0.29 0.7709

Fiber Share Introducing New

Products or Processes (CIS Eq.)484.33 83.795 5.78 <.0001

Food Probability Substantive

Innovator-146.081 52.49 -2.78 0.0057

Food Share Introducing New

Products or Processes (CIS Eq.)-110.174 52.933 -2.08 0.0383

Information Probability Substantive

Innovator412.369 76.328 5.40 <.0001

Information Share Introducing New

Products or Processes (CIS Eq.)200.25 62.53 3.20 0.0015

wojan - subject base innovation research 2014 ers rural innovation survey

Data & Analytics