MusicSense: Contextual Music Recommendation using

Emotional Allocation ModelingRui Cai, Chao Zhang, Chong Wang, Le

i Zhang, and Wei-Ying Ma

Proceedings of the 15th international conference on Multimedia, MULTIMEDIA '07

Keywords: MusicSense, contextual music recommendation, emotional allocation modeling, moods

Short papers poster session 2 - arts, content, applications

2

劇情 Introduction (1/2)

• automatically recommendation now has becoming an important role in music sales

– Last.fm - collaborative filtering

– Pandora - content similarity

• passive mode – queries -> recommendation

• active way

– Google AdSensedeliver contextual advertises matched to websites’ content

3

劇情 Introduction (2/2)

• MusicSense– contextual music recommendation– automatically deliver music pieces (or their thumbnails)

which are relevant to the context of a Web page when users read it

– measure the context relevance between music songs and Web pages

– emotion as the bridge for such a relevance matching– music is all about conveying composers’ emotions– lots of Web pages such as Weblogs also express senti

ments of writers

4

小小的 paper review

– Lu et al. utilized Gaussian mixture models to classify music songs into four emotion categories, using acoustic content features like intensity, timbre, and rhythm

– Cui et al. have comparatively studied various supervised classifiers to classify online product reviews into positive and negative opinions

– Leshed et al. tried to categorize Weblogs into ten most frequently used moods with SVM

• most works can handle only a few mood categories• difficult to collect enough high quality training data for all

possible emotions• cross-modal emotion mapping is still an open problem

5

MusicSense怎麼辦• A probabilistic modeling called Emotional Allocation t

o characterize songs and Web pages as distributions in a common emotion space– leverages both the statistics from a large-scale We

b corpus and guidance from psychological studies– keeps the inference ability of generative models

• relevance matching– songs and Web documents are respectively repres

ented by a collection of word terms, based on which their emotion distribution parameters are optimized in an iterative way

– Kullback-Liebler divergence

6

Framework

7

Three steps

– emotional allocation modeling– Web-based music description generation and

Web document analysis– probability inference and relevance matching

• refer to knowledge of psychology, to get a relatively complete and well-structured emotion vocabulary (40 basic emotion vocabulary)– R. Cowie, E. Douglas-Cowie, and et al. Emotion reco

gnition in human computer interaction. IEEE Signal Processing Magazine, 18(1):33–80, 2001

8

content-based music analysis

• lyrics and reviews to describe the semantic of a song• use search engines to retrieve more information from the Web to cha

racterize songs– we just prepare two queries in the form of “title + lyrics” and “title

+ reviews”, respectively– first page returned by the first query, and the top 20 pages return

ed by the second query– retrieved pages are merged as a virtual document after removing

HTML tags and stop words– tf × idf is computed as its weight– top N terms with the highest weights are selected out as the desc

ription (N = 100 the most informative terms)• Web documents – Blog – 100 highest tf × idf are kept as salient wor

ds

9

Emotional Allocation Modeling• Assume that

– given a language and its vocabulary– different emotions should have different distributions over t

he terms in this vocabulary– the frequencies of a term under different emotions are also

different• Given a collection of terms (e.g. a document),

– we can suppose it is generated by sampling a mixture of various emotions,

– as terms in this collection can be considered as controlled by different emotions.

• The parameters of such a sampling can be computed in a maximum likelihood manner

• In such a way, a term collection would have a certain allocation of emotions, in form of a probability distribution.

10

Model LDA 模型

k 種 emotion, m 種字

latent Dirichlet allocation

11

β

12

Relevance Matching

Distance 越短越好

13

實驗 Evaluation

• collected 100 songs and 50 Weblogs• Five college students were then invited to label

the ground truth– Each labeler was asked to listen each song and then

tag it with one or more words from the forty emotions in the Basic English Emotion Vocabulary [8]. The Weblog posts were also tagged in the same way.

– For each Weblog, each labeler was asked to find out 3 ∼ 5 songs, which are the most ideal candidates in his (her) mind for listening when reading that blog post, from all the 100 songs.

14

關於 Emotion Allocation 的結果

0.71 on this song. The average correlationcoefficient over the whole 100 songs is about 0.48

15

關於 Music Recommendation 的結果

• Merge first - on average, there are around 5.75 such suggestions for each post

• top N ranked songs are selected as recommendations

16

Conclusion music recommendation (MusicSense)

– new probabilistic model called Emotional Allocation Modeling– each song (or a Weblog) is generated with a distribution over th

e mixture of emotions– Emotion acts as a bridge for the relevance matching between b

logs and songs– Preliminary experiments

• Future– deeply investigate some current implementation details to impr

ove the performance– also try to utilize more information besides emotion to measure

the relevance between music and documents– carry out more user study to design an idea UI to deliver the co

ntextual music recommendation