In Media Res: Trends, Fads, Bubbles and Massively Scaled Analyses

Thomas Ball

Marketing Modelers Group

April 10, 2014

Highlights

• Setting the tone

• Growth is king

• Watersheds in strategic thought

• Theories of innovation and diffusion

• A few key trends in our current predicament

• Homologies between natural phenomena, finance and fashion

• Massively scaled analyses

• The arbitrage of ignorance

Setting The Tone…

Nietzsche noted that the seeds of any trend already exist, latent in the cultural sediment

Neal Gabler: “We’re a society driven by entertainment. In an entertainment culture, everything must compete with entertainment…”

Christopher Hitchens: “The pleasures and rewards of the intellect are inseparable from uncertainty, angst, conflict and even despair”

Lao Tzu, The Way of Life (Witter Bynner trans.): “Whether a man dispassionately sees to the core of life or passionately sees the surface, the core and the surface are essentially the same…”

Trends Are UbiquitousSome tentative definitions:

• A trend is a long-term or enduring influence on behavior, analogous to open ocean waves

– The Ancients had no concept of “trends” viewing existence as eternal and static

– Modern notions are typically credited as originating with Vico’s 1725 Nuova Scienza

• A fad is a short-term burst in behavior usually starting with explosive growth that rises to a single peak, followed by slower ebbing, analogous to froth on the beach from a breaker

– Bubbles are closely related to fads but are financial in nature and refer to unrealistic prices detached from intrinsic value

Wave Motion Does Not Change With Water Depth

Source: Daniel Bell, Personal communication, 2002

Froth Forms

FadsTrends

Trends And The Bottom LineHow can the analysis of trends, fads and bubbles be used to enhance business performance?

• Exploring nonlinear vs linear growth

• Can be leveraged as early warning systems for:

– Breakout ideas or new products

– Potential negative revenue surprises

– Expected timing of trends and bubbles

• Are directly relatable to key performance metrics, e.g., stock price, financials, YAG sales

• Aid in supply chain planning as analyses of this type can answer questions related to the depth of purchase orders and when to get out of a sales fad or trend for an item

– Improved predictions and forecasts: direction, magnitude, acceleration and likely ceiling

– Evaluated in terms of a prospective hit rate of actual vs predicted outcomes relative to what was previously used

• Facilitate “getting ahead of the curve” in hopes of distinguishing between smoke and mirror fads versus more durable trends in the flow of new ideas and products

– Text and image mining can be instrumental in facilitating this

Source: http://www.theworldeconomy.org/MaddisonTables/MaddisontableB-10.pdf, http://kk.org/thetechnium/archives/2008/10/the_expansion_o.php , http://smartregion.org/2011/03/creative-class Daniel Bell, The Coming of Post-Industrial Society, 1974

Neolithic ~2 million pop

1500 1560 1600 1650 1690 1725 1775 1800 1850 1885 1925 1965 2010

We find ourselves thrown into the middle of things…

Growth Is King

• Exponential growth since the Industrial Revolution (~1760+)

• The “Knowledge Society” emerged when the Services Sector eclipsed the Industrial Sector in growth (~1920s)

• The production and flow of ideas is a primary source of growth

Wealth and Population1-2010 AD

$0

$10,000

$20,000

$30,000

1 1000150016001700182018701900195019702010

0

2

4

6

Population

Year (Discontinuous)

U.S. Occupational Change 1800-2010

Million Billion

Wealth

Production of Ideas1500-2010

Year

# Books

Agriculture

Manufacturing

Low Wage Service Sector

Knowledge Workers

Millions Employed70

60

50

40

20

30

10

0

Services

Year

~Industrial Revolution

~Knowledge Society

~Industrial Revolution

Accelerating Rates Of Change

Source: Adapted from Stewart Brand, Whole Earth Review, 2000

Differential Rates of Change in Social Layers

Nature

Culture

Governance

Infrastructure

Markets/Commerce

Technology

Fashion

SOCIAL LAYERS

Cosmos

Language

Earth

In one framework society is composed of layers, each with its own rate of change

• Slower layers provide stability while faster layers drive innovation

• This hierarchical linear view, while helpful and illuminating, cannot be correct

E.g., “Competitive advantage” has been

reduced to a few months if not weeks

Acceleratin

g C

han

ge

Widespread shift towards greater uncertainty and disruptions to normative business practices

Uncertainty and The Business Landscape

Watersheds In Strategic Thought

A Clear Enough Future

What can be known?

Linear forecasts drive risk-based strategies

Intuition works well hereLinear systemsChange is gradualBehavior is deterministic and predictable

Alternate Futures

A few discrete outcomes define the future

A Range of Futures

A range of possible outcomesNo natural scenariosIntuitive, gut decisions less effective

True Uncertainty

Shrinking evidence that forecasts workNonlinear systemsExtreme changes in behavior can occur

abruptly and without warningBehavior is deterministic but not predictableLearning to live with uncertainty, doubt,

approximate or imprecise answers

From Risk to Greater Uncertainty and ComplexityLow High

1

2

3

?

• Anomalies regarding the assumptions of competitive advantage and sustainability of growth

• Trend towards hypercompetition in a widened arena of business operations versus an industry-specific focus

• Risk is known and quantifiable, uncertainty is neither

Source: Courtney and Kirkland, Strategy Under Uncertainty, HBR, 1996 Richard D’Aveni, Hypercompetition, 1994 Rita McGrath, The End of Competitive Advantage, 2013 Olivier Compte and Andrew Postlewaite, Uncertainty, Ignorance and Strategy, 2014 Kate Raworth, Royal Society for the Arts, Growth Is Not Enough, 2014

Models Of Innovation And Diffusion Theories of innovation and diffusion are rooted in analysis of nonlinear logistic growth curves

Source: Jesse Ausubel, DRAMs as Model Organisms for Study of Technological Evolution, 2001 Steven Johnson, Where Good Ideas Come From, 2010 Jonah Berger, Contagious, 2013 Alex Pentland, Social Physics : How good ideas spread, 2014

• Classic models are built up from an individual time series with models focused on a single diffusion curve based on cumulative data possessing a known origin or zero start value

• More recent theorizing focuses on extensions of the classic model to networks, social learning, flows of ideas, crowdsourcing and quantification of virtually everything – with less emphasis on the individual’s role

• So, from a “classic” innovation perspective Thomas Edison was a visionary

More recent views as the spotlight grabbing manager of a lab employing hundreds of scientists

Classic Diffusion and S-Shaped CurvesEight Generations of DRAM Chips, 1970-2000

Cu

mu

lati

ve DR

AM U

nit S

hip

me

nts (M

illio

n)

8000

6000

4000

2000

4K

16K

64K

256K

1M

4M

16M

64M

1970 1975 1980 1985 1990 1995 2000

Useful Books on Social Networks

Year

From Hierarchies To Heterarchies

A Hierarchical Structure

Heterarchical Structures

Social Networks Fractal Heterarchy

Hierarchical, top-down, Tayloristic organizational structures were ubiquitous in the 20th c with clear – if static -- lines of control, authority and division of labor

Visualizing the Shift In Social Structure

- This structure allows for much greater flexibility: job profiles overlap as talent replaces skill, the corporate “ladder” flattens out or disappears entirely, human capital flows as needed

Note: “Heterarchy” was coined in1945 by Warren McCullough, a neurophysiologist. From Wiki: A heterarchy is a system of organization where the elements of the organization are unranked (non- hierarchical) or where they possess the potential to be ranked a number of different ways. The two kinds of structure are not mutually exclusive. A heterarchy may be parallel to a hierarchy, subsumed in a hierarchy, or it may contain hierarchies. In fact, each level in a hierarchical system is composed of potentially heterarchical groupings which contains its constituent elements.

• The 21st c organization is increasingly focused on flatter networks and is labeled heterarchical

Source: Alex Pentland, Social Physics: How good ideas spread, 2014 Duncan Watts, Computational Social Science, 2013 Eli Pariser, The Fikter Bubble, 2011

Leading Us to The Many, Many Megatrends…#

Me

as

ure

me

nts

pe

r p

ers

on

pe

r u

nit

of

tim

e

Traditional Information Sources and Media:E.g., print, broad demographics,Self-reported and scanner data

Emergence of ConnectivityIPv4

GlobalizationSocial Networks

Mobile TechnologyKlondike-like Wealth Bubbles

Cross-Section Time Series

Key Demographic Trends:

- Increased urbanization

- Birth rates decline

- Aging of the population

- Growth in disposable income

creates a world middle class

As well as many unknowns, uncertainties and questions with no current answers…

Singularity?Sustainability?

Time

Today? Internet of ThingsUbiquitous Computing

HyperDataLiving Laboratories

Wearable Tech

HyperconnectivityIPv6

Nano-Monetization Increasing Complexity/Fragility

Quantified Selves, Cities, SocietiesSurveillance and Control in the Panopticon

43%

13%

23%

33%

14%

3.4%

20%

51%

0

25

50

75

100

1900 1920 1935 1950 1960 1975 1985 1995 2000 2012

Trade-offs in the allocation of household expenditures are a Darwinian, zero-sum game

Trends In The US Household Budget

Sources: US Bureau of Economic Analysis, 2008, http://www.bls.gov/opub/uscs/home.htm, the Census Bureau revised Stat Abstracts after 2008 making pre-1990 information less accessible *All Other expenditures include (as % of 2012 All Other): Transportation (36%), Insurance and Pensions (22%), Healthcare (14%), Entertainment (10%), Religion and Charities (8%), Personal Care (2%), Alcohol (2%), Tobacco (1%), Miscellaneous All Other (1.7%)

US Household Expenditures by Category% of total expenditures, 1900-2012, discontinuous years, current $

%

All Other* $3.3 trill

Housing $2.1 trill

Food $820.4 bill

Apparel $215.6 bill

% Change In Hhold Expenditures

1900-2012

+151%

-76%

+41%

-70%

Necessities $3.1 trill

49% of Total -39%

• Along with significant shifts between categories since the turn of the 20 th c, expenditures on necessities – housing, food and apparel -- saw a large decline (-39%) as a percent of total expenditures with a corresponding shift into the All Other* category

All Other* $3.3 trill

51% of Total +151%

Year (Discontinuous)

From Linear to Nonlinear Models and Assumptions

Technology played a key role in this trend with the adoption of, e.g., telephones, radio and TVs as well as increasing transportation options

• This trend flattens out in the late 80s

Average Household Size in the US1890-2010

Source: US Statistical Abstracts, http://hypertextbook.com/facts/2006/StaceyJohnson.shtml http://www.nationmaster.com/graph/peo_ave_siz_of_hou-people-average-size-of-households

Household Size Shrank 50% In The Past Century

2

3

4

5

1890 1940 1970 1995 2000

2.12.22.22.2

2.32.3

2.42.4

2.52.52.5

2.62.62.6

2.72.8

3.1

0 1 2 3

Sweden Denmark

Norway Germany

SwitzerlandHolland

BelgiumBritain

France Finland Austria

USA Australia

Canada Italy

Japan Ireland

Average Household Size in the Developed World

~2010

Avg=2.5

Year (Discontinuous)

Average # People

Women are working more and spending less time with their families while the opposite is true for men

• Free time as a percent of total time has contracted since the 60s

Where Did The Time Go?

Source: University of Maryland, Scientific Research on the Internet, Base=168 hours per week

% Utilization of Total Time2000

Overall % Change1965-2000

% Change: Men vs Women1965-2000

% Women % Men

16%

19%

22%

44%

0% 20% 40%

Family

Free Time

Work

Personal Care,Sleep

-10.5%

-1.4%

11.9%

-0.2%

-15% -5% 5% 15% 25%

55%

5%

-14%

-2%

-27%

-8%

104%

1%

-40% -10% 20% 50% 80% 110%

%%

A less active, more passive American emerged along with the Internet as dramatic declines are to be seen in arts attendance and leisure pursuits from pre-web days to the present

• An overall metric of adult arts attendance declined 18% from 1982 to 2008 from 39% to 33%

- Exercise and Volunteering are the only categories that show increases in participation

*Arts activities tracked since 1982 are attendance at jazz, classical music, opera, musical plays, non-musical plays, ballet performances, and visits to art museums or art galleriesSource: NEA, Survey of Public Participation in the Arts, 1982-2008

Did The Internet Change Behavior?

Participation In Leisure Activities% of Adults, 2008**

26

28

31

32

33

42

53

0 15 30 45 60

Playing Sports

Outdoor Activities

Sports Events

Volunteer/Charity

Overall

Gardening

Exercise

-33%

-22%

-36%

14%

-18%

-31%

4%

-50% -40% -30% -20% -10% 0% 10% 20%

Total % Change1982-2008

IncreasedDeclined

Foraging, Exploration And EngagementAs hives grow or food sources decline, bees will migrate to new locations

• Bees alternate between random foraging and new hive spotting

• Once a bee finds a good possibility, they return to the hive and perform a "waggle" dance indicating the precise direction and distance of the new location

• Other bees watch the dance and elect to propagate that source

How Do Bees Do It? Is There A Human Analogue?

• New hive location is a kind of exploratory market lottery distinguished by a collaborative and decentralized process leveraging bees unique ability to find nectar

• At the risk of anthropomorphizing distinctly nonhuman behavior, the analogy might be to long-run, aggregate human behavior

• Human creativity is the wild card in the survival (or not) of the species

Source: Thomas Seeley, The Wisdom of the Hive: The Social Physiology of Honey Bee Colonies, 1996

Modeling Fads And Explosive Self-Generating DemandPredictive modeling of behaviors from phenomena such as the explosive flow of water out of a breached dam to search activity for the term “Justin Bieber”

• An inverse t-distribution fits both phenomena (ex Bieber’s burstiness)

Water Flowing From A Breached Dam vs Justin Bieber Keyword Search

Source: Google Trends, Marcch 17, 2014 “Justin Bieber” as a keyword exploded in the 12 months from May 2009 to May 2010 but did not keep pace with the growth in overall Google search activity after. GT output is normalized to mask the real, underlying raw numbers. This process creates a relative, dimensionless, opaque, ipsative metric that ranges between 0 and 100 where “0” does not mean “no activity” but activity below an unknown, inconsistent and arbitrary level that GT does not report and “100” refers to the maximum level of keyword activity for the time window after normalization. The results can and do change constantly for any number of reasons. All of this makes firm interpretation of any GT chart impossible. While one can assume that a rising trend shows increase, a down trend does not imply decline. What one can infer from a down trend is that keyword use has not grown at the same rate as the denominator (all Google search activity). The burstiness is driven by discrete events with wide media coverage in his celebrity/notoriety.

0

25

50

75

100

May-09 Nov-09 May-10 Nov-10 May-11 Nov-11 May-12 Nov-12 May-13 Nov-13

“Ju

sti

n B

ieb

er”

Ke

yw

ord

Financial Bubbles: Tulip Mania In Renaissance HollandA price bubble occurred during the 17th c Dutch Renaissance or “Golden Age” involving speculation in tulip bulbs• Accounts vary but one sale of 40 bulbs was recorded at 100,000 florins

- To put that in mid-17th c Dutch perspective, a ton of butter cost around 100 florins, laborers might earn 150 florins a year and "eight fat swine" cost 240 florins

- One story has it that someone cut up a rare tulip bulb, not his own, like a shallot for his breakfast, leaving its real owner apoplectic with rage

Dutch Tulip Bulb Prices1634-1637

Source: http://en.wikipedia.org/wiki/Tulip_mania

Financial Bubbles And Shiller’s Irrational ExuberanceThe past three decades of financial history contain several examples of the destruction in wealth that can be wreaked when asset price bubbles burst • Nobel Laureate Robert Shiller was an early whistle-blower regarding the potential ravages of the Dot Com Bubble as well as the subsequent real

estate bubble leading into the Downturn

- His book may have “created” the Dot Com Bubble recession with its nearly wholesale adoption by then Federal Reserve chairman, Alan Greenspan, pulling the rate levers

Source: Robert Shiller, Irrational Exuberance, 2001,2006, http://aida.econ.yale.edu/~shiller/data.htm

The S&P 500 vs the Consumer Price IndexAnnualized, 1871-2013

0

400

800

1,200

1,600

1871 1900 1929 1958 1987

0

100

200

Year

S&

P 5

00

CP

I, 1983=

100

CPI S&P

2013

Bubbles?

Extreme Value Models Used To Control Flood RiskMuch of Holland is below sea level and survives only due to an extensive battery of sea dikes• In the early 50s, the worst sea surge in their history far exceeded then current flood controls, killing thousands

• Using a 400+ year record of maximum annual storm surge height, Dutch mathematicians estimated the minimum required sea wall that would protect them against a 1 in 10,000 year event…the levees were then rebuilt to that specification

• With global warming, the Dutch are again rebuilding…this time for a 1 in 100,000 year event

Source: http://news.nationalgeographic.com/news/2001/08/0829_wiredutch.html D van Dantzig, Economic Decision Problems for Flood Prevention, 1956 Paul Embrechts, Claudia Kluppelberg, Modelling Extremal Events: for Insurance and Finance (Stochastic Modelling and Applied Probability ), 2012 Mary Mapes Dodge, Hans Brinker and the Silver Skates, 1865

Landscape Sculpture Commemorating the “Little Dutch Boy”In Madurodam, The Netherlands

The Importance And Relevance Of FashionFashion is one of the world's most important creative industries

• It is the major output of a global business with annual U.S. sales of more than $200 billion—larger than those of books, movies, and music combined

• Fashion has provided economic thought with canonical examples of consumption, conformity, diffusion, networks, trends and fads

• Social thinkers have long treated fashion as a window into social class, change and culture

• Cultural theorists have focused on fashion to reflect on its symbolic meaning and social ideals

• It is a greenhouse for the analysis of trends and fads

Source: Hemphill and Suk, The Law, Culture and Economics of Fashion, 2009

FEEDBACK/PROPAGATE

3 Years of Sales

Decline

“Blooming, Buzzing Confusion” in Idea FlowsDecisions Analogous to Bees and New Hive

Location

Vis

ion

ari

es

Pla

ce

Be

ts

Sources ToolsWellesP K DickBellWintourJobsDavosMeekerArtistsAcademics

Long-Tailed

Product?Growth

Maturity Decline

Decelerating Sales

Pe

rform

an

ce

Sales DataMarketing SpendCompetitive Info from

Comparison Engines

Tools

Pro

du

cti

on

of

the

Po

rtfo

lio

Market Mix ModelingData MiningPredictive ModelingNetwork and Diffusion ModelsRecommender SystemsMachine LearningTest, Learn and Refine

Sources

Introduction

Accelerating Sales

0

Innovation Pipeline Sales Curves and the Product Life Cycle Execution

Social MediaPatentsHiring TrendsFilm, Books, Art, etc.VC InvestmentsDemographics (Youth

and Agelessness)Tech ConferencesBlogs

Go – No Go

Visionaries

Visionaries Originate While Markets Imitate, Diffuse and Drive

Fragmented Market Arenas And Proliferating, Rapidly-Cycling Products Drive Need For Massive-Scale Monitoring And Analysis

Evidence-Based Decision-Making

R&D-Tacit KnowledgeText and Image MiningPrediction MarketsContinuous TrackingCompetitive Info from

Comparison Engines

La

un

ch

Time

Sc

an

, M

on

ito

r, O

rig

ina

te,

Imit

ate

Trends and fads are driven by evanescent, black box creative ferment and idea flows that are difficult to capture, quantify and predict

*Word-of-mouth and Ready-to-WearSource: Teri Agins, Personal communication, The End of Fashion, 2000 David Wolfe, Doneger Group, Personal Communication, 2014

Cascades in Fashion Industry Trends

Premiere Vision Summarizes the Latest Ideas

Paris’ fabric and textile show is a first look at latest trends based on fabric purchases

Social media WOM plays a tacit role in diffusion of the latest designs

Networks Diffuse the Latest Ideas

Runways reflect a surprising degree of consistency

Fashion Week Presents the Latest Designs

Buyer purchases play a huge role in shaping what consumers see

Retail Buyer Purchases Shape Consumer Choice

Emerge: Haute Couture

Cascade: Couture

Consolidate: Mass Luxe

Diffuse: RTW or Pret-a-Porter

Decoding Trends And Fads: Artists As Cultural AntennaH

aute

Co

utu

re a

s t

he

Av

ant-

Gar

de

Ho

w d

ec

od

e e

me

rge

nc

e a

s w

ell

as

im

po

rta

nc

e,

pro

mis

e o

r p

rete

ns

e?

Art, Innovation,

Imitation

CommoditizationMonetization

Trend Setting

Trend Is Set

Fast Fashion, e.g., Zara, H&M

Fast Fashion Skips Over Haute Phase

The evolution of Chinese fashion street styles based on image mining of thousands of pictures taken in Shanghai and Beijing in the last five years suggests:

Identifying Fads And Trends With Machine Learning Algorithms

Source: http://www.jingdaily.com/from-social-status-to-self-expression-the-rapid-evolution-of-chinas-street-style/42059/?utm_source=twitterfeed&utm_medium=twitter# Svante Jerling, Personal communication, P1.cn, 2014

 Tracking Handbags With LVMH Logo in China2008-2013

• Tastes may have shifted away from impersonal, conspicuous status statements using “logo” brands such as LVMH to more personal, “niche” brands

• Slowing economic growth as well as a crackdown on corruption and pirating are also factors

• These include tracking social mobilization and civil unrest, epidemic forecasting, real-time prediction of stock market moves, rate of picture postings on Flickr during Hurricane Sandy that correlated with the storm’s barometric pressure, networks of A-listers and their entourage

- After Currid-Halkett, fashion is part of the celebrity network and could be decoded as such

Leveraging Social Media To Predict Emergent PhenomenaOpen Source Indicators (OSIs) such as text and image mining of social media have seen wide use in the prediction of emergent social phenomena

The Fame Game: Celebrity Networks

Source: IARPA program on OSIs, enter these search terms into a Google Scholar search window: D12PC00337 OR D12PC00285 OR D12PC00347 Elizabeth Currid-Halkett, Starstruck: The Business of Celebrity, 2012, she posits five tiers in the celebrity system: first, celebrities and aspirants, second, PR reps, agents and handlers working directly for the first tier, third, the supporting machinery of lawyers, chauffeurs, bodyguards, couriers and attendants, fourth, “preppers,” e.g., stylists, beauty salons and fifth, media

The Arbitrage Of IgnoranceRediscovering the value inherent in ignorance, uncertainty, diffidence, cultivation of doubt, error and insecurity as modes of learning, motivation and discovery

We’re All Drinking From Fire Hoses Now

Source: Stuart Firestein, Ignorance: How it drives Science , 2012 Rita McGrath, The End of Competitive Advantage, 2012

• Sherlock is dead! Long live Sherlock!

– It isn’t possible to keep up with everything. Who was the last intellectual that could?

• Hype, hubris and disinformation in massive quantities of information: our modern Babbits

• Approximation versus false precision

– What does it mean to “optimize” inaccurate and incomplete data?

– Can some grad student in decision theoretics develop a “Law of Bad Data?”

• In the midst of a paradigm shift

– Disruptions are everywhere

– Anomalies in Porterian assumptions of the sustainability of growth

Appendix

Vulnerability, Blind Spot

Enhanced Johari’s WindowKnown to Self Not Known to Self

Known to Others

Not Known to

Others

Façade/Insight

Johari’s Window

Hidden, Façade, Private

or Tacit Knowledge,Advantage

Unknown to Self or Others,

Area with Greatest Potential

Knowledge Ignorance

Developed in the 50s as a framework visualizing the structure of interpersonal knowledge

• Still sees wide use in corporate training events

• Adapting and extending the original, symmetric boxes to the wider arena of what is known vs not known, a potentially more representative and asymmetric framework emerges

Source: JosephLuft and Harrington Ingham, The Johari window, a graphic model of interpersonal awareness, 1955 Lowell Bryan, McKinsey Director, “~80% of a corporation’s knowledge assets are tacit,” 2004 Associate Lunch talk

Johari’s Window

Open Arena, Mano a Mano,

Trench Warfare

Hic Leonem,Uncertainty,

Black Swans, Exploration,Serendipity,

Greatest Potential,Generally Not Quantifiable

But Not Unknowable

Competitive Arena

Advantage, Private,Tacit,

Façade, Insight

Vulnerability, Blind Spot