intuition vs. metrics
TRANSCRIPT
![Page 1: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/1.jpg)
Intuition vs. Metricsh i b tThe Big Debate
Andrew Mo | Playfish China
![Page 2: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/2.jpg)
changing how the world plays games2
changing how the world plays games
![Page 3: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/3.jpg)
![Page 4: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/4.jpg)
![Page 5: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/5.jpg)
INTUITION
“I will quit before“I will quit beforeI make a gamebased on analytics”
– Warren Spector, Disney
![Page 6: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/6.jpg)
METRICS
“Creating SuccessfulCreating SuccessfulSocial Games:U d t di Pl Understanding Player Behavior”
– Mark Skaggs, Zynga
![Page 7: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/7.jpg)
which one wins?
vs
![Page 8: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/8.jpg)
(drum roll)(drum roll)
![Page 9: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/9.jpg)
both.both.
![Page 10: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/10.jpg)
thank you … questions?thank you … questions?
![Page 11: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/11.jpg)
the (actual) plan
• why do we need both?
• why is it hard to do it right?• why is it hard to do it right?
• when to use which?
![Page 12: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/12.jpg)
a much wider debate…a much wider debate…
![Page 13: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/13.jpg)
inspiration analysisinspirationpatterns
analysisdata
holistichunch
intentionallogichunch
tacitlogicexplicit
![Page 14: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/14.jpg)
you need bothyou need both
![Page 15: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/15.jpg)
“Not everything that counts can be counted and not everything that can be counted counts”
![Page 16: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/16.jpg)
“R di l i ti “Radical innovation requires both evidenceand intuition: evidence and intuition: evidence to become informed, and intuition to inspire us pin imagining and creating new and better
ibiliti ”possibilities.”
— Jane Fulton Suri IDEO— Jane Fulton Suri , IDEO
![Page 17: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/17.jpg)
it’s evolution, baby.
![Page 18: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/18.jpg)
but it’s hard!but it s hard!
![Page 19: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/19.jpg)
single biggest reason for failuresingle biggest reason for failure…
…using the wrong one at the wrong timeg g g
![Page 20: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/20.jpg)
two talestwo tales
![Page 21: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/21.jpg)
tale #1: wrong numbertale #1: wrong number
![Page 22: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/22.jpg)
once upon a time, a CEO …
![Page 23: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/23.jpg)
… was all about safety
![Page 24: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/24.jpg)
the problemmetrics were “improving”
• safety targets were being metimproving being met– Initiatives underway– money spentmoney spent
• … but clear signs of … but clear signs of problems:– Numerous safety hazards
wk1 wk2 wk3 wk4
reported at supposedly "upgraded" refineries
![Page 25: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/25.jpg)
BP Deepwater Horizon Explosion (April 20, 2010)
![Page 26: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/26.jpg)
tale #2: trust metale #2: trust me
![Page 27: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/27.jpg)
once upon a time, a CEO …
![Page 28: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/28.jpg)
… was all about money
![Page 29: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/29.jpg)
a big believer in intuition
• courses for executives “on trusting their instincts when making difficult decisions”
![Page 30: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/30.jpg)
the problem
• intuition was trusted even when key data said otherwise
• poor decisions left the company d t l t f i kexposed to a lot of risk
![Page 31: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/31.jpg)
![Page 32: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/32.jpg)
when to use which?when to use which?
![Page 33: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/33.jpg)
choosing the right one
• your values matter
• your team matters• your team matters
• the problem matters
![Page 34: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/34.jpg)
values mattervalues matter
![Page 35: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/35.jpg)
designer proposed this
![Page 36: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/36.jpg)
pm proposed this
![Page 37: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/37.jpg)
went to the VP with this
![Page 38: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/38.jpg)
data driven values
…ultimately tested 44 different shades of blue
![Page 39: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/39.jpg)
![Page 40: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/40.jpg)
team mattersteam matters
![Page 41: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/41.jpg)
traditional game developmentProducer(Creative)
Game Designer(Creative)( ) ( )
web developmentD i P d t MDesigner
(Creative)Product Manager
(Data)
social game developmentsocial game developmentProducer/Designer
(Creative)Product Manager
(Data)(Creative) (Data)
![Page 42: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/42.jpg)
![Page 43: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/43.jpg)
balance of power is key
Producer Product ManagerProducer(Creative)
Product Manager(Data)
![Page 44: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/44.jpg)
the problem mattersthe problem matters
![Page 45: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/45.jpg)
Intuition versus Analysis: Strategy & Experience in Complex Everyday Problem Solving*
hi hl l i t i t itihighly complex inputs intuition
less complex inputs analysis
* Pretz, J.E. (2008)
![Page 46: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/46.jpg)
more complex intuition
![Page 47: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/47.jpg)
less complex analysis
…ultimately tested 44 different shades of blue
![Page 48: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/48.jpg)
intuition best practicesintuition best practices
![Page 49: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/49.jpg)
![Page 50: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/50.jpg)
![Page 51: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/51.jpg)
“The analysis, based on past on past events, certainly seems sound. But your instincts are the only way you're going to do something do something unsound.
And unsound things become hits. Sound ones never do.”
Seth Godin— Seth Godin
![Page 52: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/52.jpg)
how to use intuition
• experience is important• lean towards it for complex
problems not for detailsproblems, not for details• lean towards it for long-term planning• lean towards it when creating
something newsomething new• back it up with data whenever possible
![Page 53: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/53.jpg)
Need an example of clearing false assumptions
metrics best practicesmetrics best practices
![Page 54: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/54.jpg)
![Page 55: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/55.jpg)
![Page 56: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/56.jpg)
good things about metrics
• what gets measured gets done• good metrics drive good behaviour• good metrics help you make good • good metrics help you make good
decisions• good for highly measurable
environments with short feedback loopsenvironments with short feedback loops
![Page 57: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/57.jpg)
bad things about metrics
• what gets measured gets done• bad metrics drive bad behaviour• bad metrics help you make bad • bad metrics help you make bad
decisions• bad for noisy environments with long
feedback loopsfeedback loops• metrics can age quickly
![Page 58: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/58.jpg)
metrics best practices
• align with your goals• good enough (vs. perfect)• keep it simple a few good ones• keep it simple … a few good ones• revise, revise, revise
… things change; don’t be afraid to add, change and removeadd, change and remove
![Page 59: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/59.jpg)
closing commentsclosing comments
![Page 60: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/60.jpg)
final words
• you need both!– metrics are good to optimise and to clear
false assumptionsp– intuition can lead you out of your rut (for
good and for worse)good and for worse)– Trust your gut and prove it with data!
th i ht t th i ht ti b t • use the right one at the right time, but don’t hide behind either
![Page 61: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/61.jpg)
![Page 62: Intuition vs. Metrics](https://reader036.vdocuments.site/reader036/viewer/2022071601/613d3033736caf36b75a5d07/html5/thumbnails/62.jpg)
thank you … questions?thank you … questions?