Download - Data in development @ Spotify
![Page 1: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/1.jpg)
Data in developmentOscar Carlsson Data Scientist @ Spotify
![Page 2: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/2.jpg)
![Page 3: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/3.jpg)
![Page 4: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/4.jpg)
Move fast!
No decision making bottlenecks
Every squad should be autonomous
No dependencies
![Page 5: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/5.jpg)
Product Owner
QA
Client dev(s)
Backend dev(s)
Designer
Data Analyst
![Page 6: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/6.jpg)
Playlist Your Music
Audio playback
Göteborg
![Page 7: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/7.jpg)
Traditional development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release it
![Page 8: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/8.jpg)
Traditional development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release it
“This seems
reasonable to do”
“I would really
like this feature”
![Page 9: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/9.jpg)
Traditional development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release it
What if the idea
isn’t useful to users?
Nobody uses this.
Who came up
with this idea?
![Page 10: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/10.jpg)
Are we representative of our users?
totally fictional graph to illustrate that employees who use the product is probably not representative of the normal users!
Ratio
of u
sers
0
0,225
0,45
0,675
0,9
Engagement score
0 1 2 3 4 5 6 7
Normal users Employees
![Page 11: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/11.jpg)
Data to the rescue!
![Page 12: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/12.jpg)
1. Analyse the data to get insights on what is
useful for real users
![Page 13: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/13.jpg)
A little more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release itInsights
![Page 14: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/14.jpg)
A little more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release itInsights
Probably something that is useful
![Page 15: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/15.jpg)
A little more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release itInsights
Probably something that is useful This particular solution works.
But is it the best?
![Page 16: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/16.jpg)
A little more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Code it Release itInsights
Are we sure what we are
working on solving the problem we saw?
![Page 17: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/17.jpg)
A little more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Release itInsights Test TestPrototype
One platform only?
Think it Code it Release itInsights
Code it
![Page 18: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/18.jpg)
How to test this? A/B testing!
![Page 19: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/19.jpg)
A/B testing
Statistical hypothesis testing
Simple randomized experiment with >= 2 variants (A, B)
![Page 20: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/20.jpg)
A/B testing
Control A B
“The shuffle button test”
Hypothesis: From insights X we believe redesigning the shuffle button will increase number of users who click on it
![Page 21: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/21.jpg)
A/B testing Reach
Shar
e of
use
rs w
ho p
ress
sh
uffle
(%)
0
10
20
30
40
50
Control A B
Control A B
Fictional numbers!
![Page 22: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/22.jpg)
A/B testing Reach
Shar
e of
use
rs w
ho p
ress
sh
uffle
(%)
0
10
20
30
40
50
Control A B
Control A B
Depth
Clic
ks p
er u
ser/
day
0
1
2
3
4
5
Control A B
Fictional numbers!
![Page 23: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/23.jpg)
A/B testing Reach
Shar
e of
use
rs w
ho p
ress
sh
uffle
(%)
0
10
20
30
40
50
Control A B
Control A B
Depth
Clic
ks p
er u
ser/
day
0
1
2
3
4
5
Control A B
Retention
Ratio
of u
sers
that
clic
k ag
ain
0
3,75
7,5
11,25
15
Control A B
Fictional numbers!
![Page 24: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/24.jpg)
2. Measure and test if what you are doing is
solving what you hoped to
![Page 25: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/25.jpg)
Even more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Release itInsights Test TestPrototype Code it
Maybe do something else
![Page 26: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/26.jpg)
Even more data-driven development
illustration courtesy of Dan McKinley, Etsy
Think it Release itInsights Test TestPrototype Code it
Data Data Data
![Page 27: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/27.jpg)
All of this requires collection of data
and users to test on
![Page 28: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/28.jpg)
Data Measures from the clients and services
User surveys
User interviews
![Page 29: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/29.jpg)
Measures from the clients and services > 30 TB per day
Event logs
2000 machine Hadoop cluster (moving to GCP)
![Page 30: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/30.jpg)
Playlist and Your Music squad
![Page 31: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/31.jpg)
![Page 32: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/32.jpg)
More than
2 billion playlists
![Page 33: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/33.jpg)
Less users
create playlist
on iOS
Fictional insights that could be presented to the Playlist squad
Some users don’t
invest time in
creating playlists
Revisit
playlist flow
on iOS
Pre curated
playlists
![Page 34: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/34.jpg)
Audio playback squad
![Page 35: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/35.jpg)
30 million songs to
75 million users in
59 countries
![Page 36: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/36.jpg)
![Page 37: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/37.jpg)
Swedish data plans are
enormous compared to
countries X, Y,Z
Fictional insights that could be presented to the Audio playback squad
Some users only
have internet at home
but want to
stream everywhere
Offline, caching?
Improve
Offline
Experience
![Page 38: Data in development @ Spotify](https://reader031.vdocuments.site/reader031/viewer/2022021420/5880e6a01a28ab0d358b5af5/html5/thumbnails/38.jpg)
Summary• Incorporate data in all steps of development and in organisation
• Cross functional squads - feature teams
• Let Data/Insights drive ideas • A/B tests to evaluate solutions