big data, big changes: data-driven product development at etsy
TRANSCRIPT
Big Data, Big Changes: Data-Driven Product Development at Etsy
Strata London, 2012
Dr. Jason DavisDirector of Search & [email protected] / @jasondavis
Monday, October 1, 12
(The change)
Monday, October 1, 12
6 months from inception to launch3 engineering teams3 product launchesDozens of experimentsLots and lots of data analysis
Monday, October 1, 12
The switch to relevancy
Search, advertising, analytics, and more
Building big data products at Etsy
Invest in tooling
Understand tradeoffs
Appreciate complexity
Monday, October 1, 12
Monday, October 1, 12
By the Numbers
1.4B pageviews/month
$525MM sales in 2011
40MM unique visitors/month
800k shops / 150 countries
Monday, October 1, 12
SELECT * FROM listings WHERE title LIKE '%silver%' ORDER BY creation_tszDESC;
Monday, October 1, 12
Solving the recency problem
Monday, October 1, 12
Better buyer experience
Improved search relevancy
Monday, October 1, 12
Improved advertising
Search Ads
Monday, October 1, 12
More search insight for sellers
Shop Stats
Monday, October 1, 12
Building big data products
Monday, October 1, 12
Build lots of tools
Monday, October 1, 12
Go to the sourceRaw data
visit logsraw query logs
Descriptive statistics
search depthquery conversion rates
Monday, October 1, 12
Invest in tooling
Lunr
Monday, October 1, 12
Side by side testing
Monday, October 1, 12
A/B Analyzer
Monday, October 1, 12
Shop Stats with Referrals
Monday, October 1, 12
Understand tradeoffs
(most tradeoffs don’t look like this)
Monday, October 1, 12
The bike is 20% lighter!Monday, October 1, 12
Queries a, b, & c are better. Queries x & y are not.
Monday, October 1, 12
¡Hola! வணக்கம்!
Guten Tag 你好 ألّس'م عليكم
Hello नम#$ 今日は
Monday, October 1, 12
How will the change affect behavior?
Affect expectations?
User incentives?
Improved data?
Thinking short-term vs long-term
Monday, October 1, 12
Appreciate complexity
Monday, October 1, 12
Vs
Algorithmic Complexity
Monday, October 1, 12
4 months30 experiments11 wins
Success / Complexity Tradeoff
Monday, October 1, 12
June 2005
November 2006
May 2008
Today
UI Complexity
Monday, October 1, 12
Communications & Support Complexity
Monday, October 1, 12
(The change)
Monday, October 1, 12
Etsy - Strata NYCContinous Experimentation with Continuous DeplyomentSteve Mardenfeld
Scala + Cascading = ScaldingAvi Bryant
Case Study: What’s a Customer Worth?Roberto Medri
October 23-25
Monday, October 1, 12
ResourcesCode as Craft bloghttp://codeascraft.etsy.com/
Slides (including these)http://www.slideshare.net/etsy
Github http://github.com/etsy/
Join in the fun http://www.etsy.com/careers/
Monday, October 1, 12
Monday, October 1, 12