pivotal tech talk - using data to inform product decisions (22.10.14)
DESCRIPTION
Why is important to use data to inform product decisions? How can you best use data as part of the product lifecycle? This talk about the role that data can play in informing decisions (and answering questions) at different stages of the product lifecycle.TRANSCRIPT
Use data to inform product decisions
Why we do need data to inform product development
Imagine this ...
... or this
Outline
Why
Driven vs Informed
What
5 things to be mindful of
Why do we need data?
Why do we need data?
What data do we need?
It’s about asking the right questions, you fool!!
Data & the product lifecycle
What do we want?
Assumption: Sitting on a tractor all day isn’t the best use of my time
Assumption: My users want to spend less time on the tractor so that they can spend more time on other tasks
Hypothesis: We believe this is true if the users of our MVP spend 20% more time on the farm
Approaches: One Single Metric, Prototypes, MVP, Direct Observations, Competitor Analysis
Question: Is there a market need for driverless tractors?
Data & the product lifecycle
How should it work?Questions: How should people find and use their personalised TV Guide? What should they do next?
Assumption: People will expect to find their personal TV Guide under the “Guide” tab. They will then switch to the ‘full’ TV Guide to see what else is on
Hypothesis: We believe that our users will discover new content on the app if their personal guide is easier to find. We know this is true if there’s a 30% increase in click-through rate from the personal TV Guide by Dec ’14
Approaches: A/B and MVT, behavioural plan & KPIs, user stories & metrics, prototypes and user testing
Data & the product lifecycle
How is it working?
Question: Is our product / feature meeting the hypothesis?
Assumption: Our users will use this feature to create and manage their holiday plans because it is so easy to use
Hypothesis: We know that our assumption is correct if we see a 30% increase in the number of users creating new holiday plans through this new feature by December ‘14
Approaches: Usage tracking, user testing, product retrospectives and refine or reject hypothesis
Data driven
Data drivenA/B or multi-variate test continuously !
Focus on the “One Metric That Matters” !
Build hypothesis around key KPI !
Optimise your product based on data !
Are we making a noticeable difference?
BUT... What data cannot tellIs it a good product idea? !
Metrics do not always offer you the full picture !
Data is one of the factors that feed into a decision !
We typically do not own all product decisions
Data informed
Data informed
Data
Users
Intuition
Competition
Technology
Brand
Strategy
BusinessRegulation
Time
Data informedData is one of the factors to consider !
Focus on the questions that you want answered !
You cannot replace intuition or creative ideas with data !
Assess impact on relevant areas
5 things to be mindful ofFocus on asking the right questions !
Data can’t replace intuition !
Listen to the data and act accordingly! !
Build and launch with data in mind !
Be clear on hypothesis, sample size and timings
SO ...
Embrace the data, don’t fear it!
Related linkshttp://svpg.com/assessing-product-opportunities/ !http://www.romanpichler.com/blog/goal-oriented-agile-product-roadmap/
http://vimeo.com/14999991
http://www.realityisagame.com/archives/390/wooga-follows-zynga-in-metrics-driven-game-design/
http://marcabraham.wordpress.com/2013/05/03/book-review-lean-analytics/
http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions-targets/
http://marcabraham.wordpress.com/2013/09/09/some-considerations-regarding-data-driven-design/
http://insideintercom.io/the-problem-with-data-driven-decisions/
http://www.webdesignerdepot.com/2013/05/the-perils-of-ab-testing/
http://andrewchen.co/2008/09/08/how-to-measure-if-users-love-your-product-using-cohorts-and-revisit-rates/
http://codeascraft.com/2012/06/21/building-websites-with-science/