Use data to inform product decisions
Why we do need data to inform product development
Imagine this ...
... or this
Outline
Why
Driven vs Informed
What
5 things to be mindful of
Why do we need data?
Why do we need data?
What data do we need?
It’s about asking the right questions, you fool!!
Data & the product lifecycle
What do we want?
Assumption: Sitting on a tractor all day isn’t the best use of my time
Assumption: My users want to spend less time on the tractor so that they can spend more time on other tasks
Hypothesis: We believe this is true if the users of our MVP spend 20% more time on the farm
Approaches: One Single Metric, Prototypes, MVP, Direct Observations, Competitor Analysis
Question: Is there a market need for driverless tractors?
Data & the product lifecycle
How should it work?Questions: How should people find and use their personalised TV Guide? What should they do next?
Assumption: People will expect to find their personal TV Guide under the “Guide” tab. They will then switch to the ‘full’ TV Guide to see what else is on
Hypothesis: We believe that our users will discover new content on the app if their personal guide is easier to find. We know this is true if there’s a 30% increase in click-through rate from the personal TV Guide by Dec ’14
Approaches: A/B and MVT, behavioural plan & KPIs, user stories & metrics, prototypes and user testing
Data & the product lifecycle
How is it working?
Question: Is our product / feature meeting the hypothesis?
Assumption: Our users will use this feature to create and manage their holiday plans because it is so easy to use
Hypothesis: We know that our assumption is correct if we see a 30% increase in the number of users creating new holiday plans through this new feature by December ‘14
Approaches: Usage tracking, user testing, product retrospectives and refine or reject hypothesis
Data driven
Data drivenA/B or multi-variate test continuously !
Focus on the “One Metric That Matters” !
Build hypothesis around key KPI !
Optimise your product based on data !
Are we making a noticeable difference?
BUT... What data cannot tellIs it a good product idea? !
Metrics do not always offer you the full picture !
Data is one of the factors that feed into a decision !
We typically do not own all product decisions
Data informed
Data informed
Data
Users
Intuition
Competition
Technology
Brand
Strategy
BusinessRegulation
Time
Data informedData is one of the factors to consider !
Focus on the questions that you want answered !
You cannot replace intuition or creative ideas with data !
Assess impact on relevant areas
5 things to be mindful ofFocus on asking the right questions !
Data can’t replace intuition !
Listen to the data and act accordingly! !
Build and launch with data in mind !
Be clear on hypothesis, sample size and timings
SO ...
Embrace the data, don’t fear it!
Related linkshttp://svpg.com/assessing-product-opportunities/ !http://www.romanpichler.com/blog/goal-oriented-agile-product-roadmap/
http://vimeo.com/14999991
http://www.realityisagame.com/archives/390/wooga-follows-zynga-in-metrics-driven-game-design/
http://marcabraham.wordpress.com/2013/05/03/book-review-lean-analytics/
http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions-targets/
http://marcabraham.wordpress.com/2013/09/09/some-considerations-regarding-data-driven-design/
http://insideintercom.io/the-problem-with-data-driven-decisions/
http://www.webdesignerdepot.com/2013/05/the-perils-of-ab-testing/
http://andrewchen.co/2008/09/08/how-to-measure-if-users-love-your-product-using-cohorts-and-revisit-rates/
http://codeascraft.com/2012/06/21/building-websites-with-science/
