finding music with pictures: using visualization for discovery
DESCRIPTION
Slides from my SXSW 2011 talk. Here's the abstract:With so much music available, finding new music that you like can be like finding a needle in a haystack. We need new tools to help us to explore the world of music, tools that can help us separate the wheat from the chaff. In this panel we will look at how visualizations can be used to help people explore the music space and discover new, interesting music that they will like. We will look at a wide range of visualizations, from hand drawn artist maps, to highly interactive, immersive 3D environments. We'll explore a number of different visualization techniques including graphs, trees, maps, timelines and flow diagrams and we'll examine different types of music data that can contribute to a visualization. Using numerous examples drawn from commercial and research systems we'll show how visualizations are being used now to enhance music discovery and we'll demonstrate some new visualization techniques coming out of the labs that we'll find in tomorrow's music discovery applications.TRANSCRIPT
Finding Music With Pictures: Data Visualization for Discovery
Paul Lamere #musicvizThe Echo Nest
Music Discovery Challenge
2
Music Discovery Challenge
3
Listener Study
Listeners 5,000
Average Songs Per User 3,500
Percent of songs never listened to 65%
Ishkur’s Guide to Electronic Dance Music
4
Why Visualize?
5
Visualizations can engage our pattern-matching brain
Human rendered visualizations
6
Visualizations as a tree
71939
Visualizations as a tree
71939
Visualizations as a tree
71939
The greatest visualization ever?
8
Visualizations as a Map
9Dorian Lynskey - The Guardian
Section:GDN RR PaGe:8 Edition Date:060203 Edition:01 Zone: Sent at 1/2/2006 20:06 cYanmaGentaYellowblack
The Guardian | Friday February 3 2006 98 The Guardian | Friday February 3 2006
Cover story
Going underground
Could we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped
It seems like a deeply implausibleproject: to plot the history of 20thcentury music on the LondonUnderground map devised byHarry Beck in 1933. Artist SimonPatterson transformed the tube
map into a constellation of famousnames in his 1992 work The Great Bear,but he didn’t have to make them all linkup. It is, after all, a tall order to find asaint who was also a comedian. But forthis one to work every interchange hadto be logical in the context of musicalhistory, an unlikely prospect.
I started out with a packet of colouredcrayons, four sheets of A4 taped to-gether and a big box of doubt, but thedifferent character of each line quicklylent itself to a certain genre. Pop inter-sects with everything else, so that had tobe the Circle Line; classical music for themost part occupies its own sphere,which made it perfect for the DocklandsLight Railway. There were a couple offalse starts but by the end of one after-noon I had assigned genres to almost allthe lines and thrashed out most of themajor intersections. The key stationsnaturally went to the most eclecticartists, not necessarily the most impor-tant: the Beatles may be more signifi-cant than Beck but even their most de-voted fan must admit that they nevertried rapping.
The system thus in place, the nextcouple of days were devoted to writingnames in, scribbling them out (sorry,Doug E Fresh and Lynyrd Skynyrd), ago-nising over certain omissions, askingclassical music critic Tom Service for in-valuable help with the DLR, and swear-ing just a little bit. Amazingly, therewere no calamitous blind alleys. It justseemed to make sense.
I tried as far as possible to be objec-tive. Some bands I cannot stand are inhere, while some that I love dearlyaren’t. I also followed chronology wher-ever the path of the line allowed it. Eachbranch line represents a sub-genre: rocksprouts off into grunge and psychedeliawhen it reaches South-West London;hip-hop diverges, north of Camden, intoold school and New York rap. If I was re-ally lucky, the band name echoed theoriginal station name: Highbury & Is-lington became Sly & the Family Stone.
Pedants, of course, will find flaws.Musical influences are so labyrinthinethat any simple equation will be imper-fect. Where, for example, does pop stopand rock begin? How can you draw a de-cisive line between soul and funk? Theseare problems that have plagued recordshop proprietors for decades and they’renot going to be solved here. But I thinkall of these choices are justifiable giventhe limitations of the form.
Other people will quibble with omis-sions – it’s a shame, for example, thatthe Circle Line constantly runs in tan-dem with either the District or Metro-politan lines, thus leaving no room forpure pop acts such as Kylie Minogue andthe Pet Shop Boys. I should also pointout that, to keep my head from explod-ing, I limited the remit to western, pre-dominately Anglo-American music.Then there are those changes necessi-tated by London Underground’s under-standable sensitivity to explosive refer-ences: arrividerci, Massive Attack. Forsome reason, they also took exception tothe late rapper Ol’ Dirty Bastard.
But this is not some definitive historyof music. It’s an experiment to see if oneintricate network can be overlaid on acompletely different one. The eleganceand logic of Harry Beck’s design – itscombination of bustling intersections,sprawling tributaries, long, slanting tan-gents and abrupt dead ends, all suckedinto the overturned wine bottle of theCircle Line – seems to spark other con-nections and appeal to the brain’s innatedesire for patterning and structure. Plusit’s fun, as any piece of music journalismcreated with coloured crayons shouldbe. I hope you like it.
Tell us what you think of the map atwww.guardian.co.uk/artsBuy the map atwww.ltmuseumshop.co.ukSpecial thanks to Chris Townsend atTransport for London and AndrewJones at London Underground.
Music History as a London Tube Map
Visualizations as a Map
9Dorian Lynskey - The Guardian
Section:GDN RR PaGe:8 Edition Date:060203 Edition:01 Zone: Sent at 1/2/2006 20:06 cYanmaGentaYellowblack
The Guardian | Friday February 3 2006 98 The Guardian | Friday February 3 2006
Cover story
Going underground
Could we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped
It seems like a deeply implausibleproject: to plot the history of 20thcentury music on the LondonUnderground map devised byHarry Beck in 1933. Artist SimonPatterson transformed the tube
map into a constellation of famousnames in his 1992 work The Great Bear,but he didn’t have to make them all linkup. It is, after all, a tall order to find asaint who was also a comedian. But forthis one to work every interchange hadto be logical in the context of musicalhistory, an unlikely prospect.
I started out with a packet of colouredcrayons, four sheets of A4 taped to-gether and a big box of doubt, but thedifferent character of each line quicklylent itself to a certain genre. Pop inter-sects with everything else, so that had tobe the Circle Line; classical music for themost part occupies its own sphere,which made it perfect for the DocklandsLight Railway. There were a couple offalse starts but by the end of one after-noon I had assigned genres to almost allthe lines and thrashed out most of themajor intersections. The key stationsnaturally went to the most eclecticartists, not necessarily the most impor-tant: the Beatles may be more signifi-cant than Beck but even their most de-voted fan must admit that they nevertried rapping.
The system thus in place, the nextcouple of days were devoted to writingnames in, scribbling them out (sorry,Doug E Fresh and Lynyrd Skynyrd), ago-nising over certain omissions, askingclassical music critic Tom Service for in-valuable help with the DLR, and swear-ing just a little bit. Amazingly, therewere no calamitous blind alleys. It justseemed to make sense.
I tried as far as possible to be objec-tive. Some bands I cannot stand are inhere, while some that I love dearlyaren’t. I also followed chronology wher-ever the path of the line allowed it. Eachbranch line represents a sub-genre: rocksprouts off into grunge and psychedeliawhen it reaches South-West London;hip-hop diverges, north of Camden, intoold school and New York rap. If I was re-ally lucky, the band name echoed theoriginal station name: Highbury & Is-lington became Sly & the Family Stone.
Pedants, of course, will find flaws.Musical influences are so labyrinthinethat any simple equation will be imper-fect. Where, for example, does pop stopand rock begin? How can you draw a de-cisive line between soul and funk? Theseare problems that have plagued recordshop proprietors for decades and they’renot going to be solved here. But I thinkall of these choices are justifiable giventhe limitations of the form.
Other people will quibble with omis-sions – it’s a shame, for example, thatthe Circle Line constantly runs in tan-dem with either the District or Metro-politan lines, thus leaving no room forpure pop acts such as Kylie Minogue andthe Pet Shop Boys. I should also pointout that, to keep my head from explod-ing, I limited the remit to western, pre-dominately Anglo-American music.Then there are those changes necessi-tated by London Underground’s under-standable sensitivity to explosive refer-ences: arrividerci, Massive Attack. Forsome reason, they also took exception tothe late rapper Ol’ Dirty Bastard.
But this is not some definitive historyof music. It’s an experiment to see if oneintricate network can be overlaid on acompletely different one. The eleganceand logic of Harry Beck’s design – itscombination of bustling intersections,sprawling tributaries, long, slanting tan-gents and abrupt dead ends, all suckedinto the overturned wine bottle of theCircle Line – seems to spark other con-nections and appeal to the brain’s innatedesire for patterning and structure. Plusit’s fun, as any piece of music journalismcreated with coloured crayons shouldbe. I hope you like it.
Tell us what you think of the map atwww.guardian.co.uk/artsBuy the map atwww.ltmuseumshop.co.ukSpecial thanks to Chris Townsend atTransport for London and AndrewJones at London Underground.
Music History as a London Tube Map
Genre Map
10Ben Discoe http://www.washedashore.com/music/genremap.html
Finding music with pictures - SXSW 2011http://mapofmetal.com
Map of Metal
11
Visualization as a Time Series
12
Genealogy of Pop and Rock Music - Reebee Garofalo
Visualization as a Time Series
12
Genealogy of Pop and Rock Music - Reebee Garofalo
Visualization as a Time Series
12
Genealogy of Pop and Rock Music - Reebee Garofalo
Visualization as a Time Series
12
Genealogy of Pop and Rock Music - Reebee Garofalo
13
Visualizations as a graph
http://www.homeofmetal.com/
14
Seattle Band Map
http://www.seattlebandmap.com/
15
Seattle Band Map
http://www.seattlebandmap.com/
16
Hybrid visualization
franksreelreviews.comSchool of rock
16
Hybrid visualization
franksreelreviews.comSchool of rock
16
Hybrid visualization
franksreelreviews.comSchool of rock
Other types of visualizations
17
Other types of visualizations
18
Other types of visualizations
19
Other types of visualizations
20
Other types of visualizations
21
Computer rendered visualizations
22
Finding music with pictures - SXSW 2011 23
Time Series
Finding music with pictures - SXSW 2011 24
http://www.leebyron.com/what/lastfm/ - Lee Byron
Visualizing Listening History
Finding music with pictures - SXSW 2011 24
http://www.leebyron.com/what/lastfm/ - Lee Byron
Visualizing Listening History
Finding music with pictures - SXSW 2011
Maps and Graphs
25
Finding music with pictures - SXSW 2011
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.htmlDavid Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang
z
The World Of Music
26
Finding music with pictures - SXSW 2011
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.htmlDavid Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang
z
The World Of Music
26
Finding music with pictures - SXSW 2011
The Landscape of Musicby Gansner, Hu, Kobourov and Volinsky
AT&T Labs-Research
The Landscape of Music
27
Finding music with pictures - SXSW 2011
The Landscape of Musicby Gansner, Hu, Kobourov and Volinsky
AT&T Labs-Research
The Landscape of Music
27
Finding music with pictures - SXSW 2011
The Landscape of Musicby Gansner, Hu, Kobourov and Volinsky
AT&T Labs-Research
The Landscape of Music
27
Finding music with pictures - SXSW 2011
The Landscape of Musicby Gansner, Hu, Kobourov and Volinsky
AT&T Labs-Research
The Landscape of Music
27
Finding music with pictures - SXSW 2011
http://playground.last.fm/iom - Islands of Music - Elias Pampalk
Islands of Music
28
self-organizing maps
Finding music with pictures - SXSW 2011
http://www.cp.jku.at/projects/nepTune/
nepTune
29
Finding music with pictures - SXSW 2011
http://sixdegrees.hu/last.fm/index.html
Last.fm Artist Map
30
Finding music with pictures - SXSW 2011
http://sixdegrees.hu/last.fm/index.html
Last.fm Artist Map
30
Finding music with pictures - SXSW 2011
The Goal - build an artist graph suitable for music exploration
Building an artist map
31
Problem: The artist space is very complex
Finding music with pictures - SXSW 2011
The Goal - build an artist graph suitable for music exploration
Building an artist map
31
Problem: The artist space is very complex
Making a good graph
• Minimize edge crossings, bends and curves
• Maximize angular resolution, symmetry
32
Good Bad
BadGood
Cognitive measurements of graph aesthetics, Ware, Purchase, Colpoys, McGill
Finding music with pictures - SXSW 2011
digraph {
"The Beatles" -> "Electric Light Orchestra";
"The Beatles" -> "The Rutles";
"The Beatles" -> "The Dave Clark Five";
"The Beatles" -> "The Tremeloes";
"The Beatles" -> "Manfred Mann";
"The Beatles" -> "The Lovin' Spoonful";
"The Beatles" -> "The Rolling Stones";
"The Beatles" -> "Creedence Clearwater Revival";
"The Beatles" -> "Bee Gees";
"The Beatles" -> "the Hollies";
"The Beatles" -> "Badfinger";
"The Beatles" -> "Paul McCartney";
"The Beatles" -> "Julian Lennon";
"The Beatles" -> "George Harrison";
"The Beatles" -> "John Lennon";
}
Graphing the Beatles Neighborhood
33http://www.graphviz.org/
Finding music with pictures - SXSW 2011 Already interconnections start to overwhelm
Artist map - level 1 out
34
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
Artist map - level 1 in/out
35
Finding music with pictures - SXSW 2011
116 artists and 665 edges. We are totally overwhelmed.
Artist map - level 2
36
Finding music with pictures - SXSW 2011
116 artists and 665 edges. We are totally overwhelmed.
Artist map - level 2
36
Finding music with pictures - SXSW 2011
Eliminate the edges and use spring force embedding to position the artists
Simplifying the graph
37
Finding music with pictures - SXSW 2011
Eliminate the edges and use spring force embedding to position the artists
Simplifying the graph
37
Finding music with pictures - SXSW 2011
Attach artists to only one most similar artist
Reducing the edges
38
Finding music with pictures - SXSW 2011
Attach artists to only one most similar artist
Reducing the edges
38
Finding music with pictures - SXSW 2011
Attach artists to only one most similar artist
Reducing the edges
38
Finding music with pictures - SXSW 2011
Attach artists to only one most similar artist
Reducing the edges
38
wtf?
Finding music with pictures - SXSW 2011
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
39
Finding music with pictures - SXSW 2011
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
39
Finding music with pictures - SXSW 2011
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
39
Finding music with pictures - SXSW 2011
Graph is not 100% connected.
One more problem ...
40
Finding music with pictures - SXSW 2011
Graph is not 100% connected.
One more problem ...
40
Finding music with pictures - SXSW 2011
Build a minimum spanning tree from the graph
Making the graph connected
41
Finding music with pictures - SXSW 2011
Build a minimum spanning tree from the graph
Making the graph connected
41
Finding music with pictures - SXSW 2011
A browsing network for music discovery
2K artists
42
Finding music with pictures - SXSW 2011
A browsing network for music discovery
2K artists
42
Finding music with pictures - SXSW 2011
Mirrored Radio dial - by thomwatson
Interactive
43
Finding music with pictures - SXSW 2011
The SXSW Music Maze
44http://static.echonest.com/SxswMusicMaze/SxswMusicMaze.html
Finding music with pictures - SXSW 2011
Flow List
46http://bowr.de/flowlist/flowlist.html
Finding music with pictures - SXSW 2011
MoodLogic
47
Finding music with pictures - SXSW 2011
http://mpac.ee.ntu.edu.tw/~yihsuan/pub/MM08_MrEmo.pdf
Mr. Emo: Music Retrieval in the Emotional Plane
48
Finding music with pictures - SXSW 2011 http://thesis.flyingpudding.com/ - Anita Lillie
Music Box
49
Finding music with pictures - SXSW 2011
The Mufin Player
3D Track similarity
50
Finding music with pictures - SXSW 2011
Discovr
51
Finding music with pictures - SXSW 2011
www.hitlantis.com
Hitlantis
52
Finding music with pictures - SXSW 2011
SongExplorer: Exploring Large Musical Databases Using a Tabletop InterfaceCarles F. Julià - Music Technology Group - Universitat Pompeu Fabra
SongExplorer
53
Finding music with pictures - SXSW 2011
http://mediacast.sun.com/share/plamere/sitm_final.pdf - Lamere, Eck
Search Inside the Music
54