Pareto-based Science:Basic Principles—and Beyond

Bill McKelvey-----

Adelphi Conference:Social Entrepreneurship, System Thinking & Complexity

2008

Order, Chaos, Emergence

Initial condition

1st critical value:Edge ofOrder

Order

Emergence

2nd critical value:Edge ofChaos

Initial condition

1st critical value:Edge ofOrder

Order

Region of

Emergence

PowerLaws

ScaleFree

Theories

Emergence

2nd critical value:Edge ofChaos

Order, Chaos, Emergence

Fractals

CatastropheTheory &AttractorBasins

Chaos

From Fractal to Power Law

A power law is a relationship in which one quantity A is proportional

to another B taken to some power n; that is, A~Bn

Size (florets)

Fre

quen

cy

The Romanesco broccolo power law

19

80

1000

Size (log scale

Fre

quen

cy (

log

scal

e)

1

9

80

300300

Italian Income Distribution

Only the Straight line is a Power Law Distribution

Minimum amounts of: 1. Social background,2. Education,3. Personality type,4. Technical ability,5. Communication skills6. Motivation,7. Right place-right time,8. Willing to take risks

Self-Organised Criticality: The Sand-Pile Model (Bak & Chen, 1992)

Log of frequency of avalanches

Log of size of

avalanches

Log of Event Size

Logof

EventFrequency

GaussianWorld

Mean

Paretian World

Power law Inverse Slope

Mosquitoes

Elephants

Some 1st Principles of Pareto-based Science Principle #1: Given Connectivity, R/Fs Dominate Principle #2: Tension Exacerbates Connectivity Effects

– 1st Critical Value; Tension in a Teapot; Bose-Einstein Condensate– Fishnets; power grids– Fear & Greed in the Stock Market loss of heterogeneity market collapse– Business problems more connections via phone, meetings, Internet, etc. – Supply/demand-based tension hub & spoke airports connectivity of storm effects

Principle #3: Connectivity Exacerbates Tension Effects– Mini-ice age migration conflicts & black plague– LTCM; Increasing connectivity of losses & liabilities sub-prime meltdown– Traffic jams more traffic on other roads more tension– Connectivity contagion bursts pandemics– Rioters with cell phones more trouble for the police

Principle #4: The Law of Large Numbers Finds Rank/Frequency and Not Normal Distributions – A. Connectivity Replaces i.i.d.– B. Pareto Rank/Frequencies Replace Normal Distributions

Log of Event Size

Logof

EventFrequency

GaussianWorld

Mean

Paretian World

Inverse Power Law Slope

Mosquitoes

Elephants

Principle #5: Rank/Frequencies Pareto-based Methods– What is Common to Both? DNA, RNA, Genes, Organelles, Cells, Organelles, Blood– What is Different? Different Ecologies Adaptation and Species Differences

Pareto-based Method Implications

1: Need to Develop Methods for Studying Emergence 2: Studying Extremes at N = 1: “Talking Pigs” 3: Likelihood of Overlapping i.i.d. & Idiosyncratic Micro-niches

at Upper-left--i. e., Anderson’s Long Tail 4: Vertical Slices Progressing toward Smaller Samples to N = 1 5: Horizontal Scalability Dynamics Figure 6: Bak’s Self-organized Criticality--Research how Butterfly-

events Do or Don’t Scale up from Left to Right 7. Power laws as the “Diagonal” in Gini Coef. Methods 8: Power laws as Indicators of Efficaciously Adaptive Self-

organization 9. Methods aimed at Better Indicating/Locating i.i.d vs.

Connectivity Effects at intra- and inter-firm, industry and economy levels of analysis

Improving N = 1 Methods

Hermeneutics– Principle of Charity– Coherence Theory

Abduction Needed Improvements

– Few cases; same biased observer? No!

– Few cases + few diverse observers… Yes!

– When Induction doesn’t lead to Deduction…

– Scalability sensitivity to butterfly events & levers

– Extreme statistics– PL slope as criterion

variable

N = 100s to 1000s

MODEL

Multiple Observers

Log of Event Size

Logof

EventFrequency

GaussianWorld

Mean

Paretian World

Power law Inverse Slope

9: EcoSystem Research 10. Industry and Firm Structures

– Iansiti & Levien: Software ecosystem– Ishikawa: 2-digit SIC-code industries

» Power laws in “empty” categories» Other distributions in “full” categories

– Transition economies in Eastern Europe– Power law evidence of self-organization dynamics

Ma&Paor

Tesco

Wal-MartEcoSystem

Quick Examples of Missing the Initiating Events

FBI– Filling in the Dots– 52 Clues Known in Advance– Behind on the Patterns

Enron– Creative Accounting; Complicit CPA– People Knew; Memos were Sent– Behind on the Patterns

NASA– Challenger and Columbia Disasters– All Sorts of Clues about “Almost” Failures– Behind on the Patterns

Doctor in UK– Murdered over 250 patients (they think 280+)– Prescribed drugs; murdered patients; kept drugs for his “habit”– What he was doing was “known” before he killed the 1st person!

Well Performing Economies

U.S./Japan line

India/China line

Not So Hot Economies

Is UK Broken?

India

U.S.

UK

Bulgaria

BangladeshMexico

Originals in Red; Next in White; Newest in Black

Germany

Malta

Spain

Hungary

UK

Czech Rep.

Cyprus

Microsoft’s Software Ecosystem

Systems Integrators 7,752 Unsegmented resellers 290

Development services companies 5,747 Media stores 238

Campus resellers 4,743 Mass merchants 220

Independent software vendors 3,817 Outbound software firms 160

Trainers 2,717 Computer superstores 51

Breadth value-added resellers 2,580 Application service provider aggregators 50

Small specialty firms 2,252 E-tailers 46

Top value-added resellers 2,156 Office superstores 13

Hosting service providers 1,379 General aggregators; Warehouse club stores 7, 7

Internet service providers 1,253 Niche specialty stores; Sub-distributors 6, 6

Business consultants 938 Applications integrators 5

Software support companies 675 Microsoft direct resellers 2

Outbound hardware firms 653 Microsoft direct outlets 1

Consumer electronics companies 467 Network equip. & service providers, 1, 1

M. Iansiti & R. Levien 2004. Strategy as Ecology. Harvard Business Review, 2004, pp. 68–78.

Software Power Law Distribution

1

10

100

1 10 100 1000 10000

Num

ber o

f Com

pani

es

Microsoft Domains Ranked by Size

1992 2002

Per Bak’s “Avalanche” research dates back to 1987!

• Sand Grains of Irregular Shape• Some Kind of Connectivity• Critical Slope• Avalanches; Heteroscedasticity• Pareto Distribution; Power Law• Unstable Means; (nearly) Infinite Variance• Widened Confidence Intervals

• Independence Among Data Points• Approximating marbles (rounded)• Linearity• Homoscedasticity• Normal Distribution• Stable Mean; Finite Variance• Narrowed Confidence Intervals

M & M Science