compmusic workshop ii: motivic analysis and its relevance...

CompMusic Workshop II: Motivic Analysis and itsRelevance to Raga Identity in Carnatic Music

Vignesh Ishwar, Ashwin Bellur and Hema A MurthyDepartment of Computer Science and Engineering, IIT Madras

e-mail: [email protected]

Indian Institute of Technology,Madras.

13 July 2012, Istanbul, Turkey

Introduction Database Phrases Motif Identification Phrase Recognition Distinguishing Phrases Future Research

Outline

Introduction

Database

Phrases

Motif Identification

Phrase Recognition

Distinguishing Phrases

Future Research

Slide 2/21


Introduction

• Necessity of Motivic Analysis

0 100 200 300 400 500100

150

200

250

Bhairavi Phrase 1

0 500 1000 1500100

150

200

250

Bhairavi Phrase 2

0 50 100 150 200 250150

200

250

300Kamboji Phrase 1

0 100 200 300 400100

200

300Kamboji Phrase 2

0 100 200 300 400

200

300

400Kamboji Phrase 3

0 200 400 600 800 1000 1200

200

300

400Sankarabharana Phrase 1

0 50 100 150

200

300

400Varali Phrase 1

0 50 100 150 200 250 300 350

200

300

400Kalyani Phrase 10 0.2 0.4 0.6 0.8 1

0

0.5

1

MACHINELEARNING

Figure :Slide 3/21


Database

• A Database of 1 TeraByte of Carnatic Music Concerts was mined.• Occurrences of 35 rAgas in the database were monitored.• The rAgas chosen for analysis being:

• tOdi - 1797 occurrences.• kalyAni - 1712 occurrences.• kAmbOji - 1622 occurrences.• bhairavI - 1384 occurrences.• sankarAbharanA - 1206 occurrences.• varALi - 483 occurrences.

• rAgas across various artists, male and female, and instrumental werechosen for analysis.

• tOdi being a complex rAga, has scope for analysis of its own, and hence,has been excluded from this set.

• Tha AlApanA section of the piece in that rAga was used for labelling ofphrases.

Slide 4/21


Attempt to quantize phrases into gamakAs

0 500 1000 1500120

140

160

180

200

220

240

Number of Gamakas in a Phrase

Gamaka1 Gamaka2 Gamaka3

Figure : Number of gamakAs in a Phrase

Slide 5/21


The Next Step : Labelling Motifs

• rAgas and characteristic motifs.• These characteristic phrases were identified by a professional musician.• Table below shows the number of phrases labelled for each rAga with the

number of instances of each phrase across the phrases.

rAga Name Phrases labelled Instances ArtistsbhairavI 10 205 20kalyAni 9 138 12kAmbOji 30 343 20sankarAbharanA 10 366 20varALi 5 144 5

Table : Total number of phrases

Slide 6/21


• Number of phrases, as seen above, are more. However, number ofoccurrences are meagre.

• Machine learning algorithms require a substantial number of occurrencesfor training and testing of a single phrase.

• Based on number of occurrences, the following phrases were chosen forbuilding models.

rAga Phrases InstancesbhairavI Phrase 1 52

Phrase 2 70kalyAni Phrase 1 52kAmbOji Phrase 1 104

Phrase 2 49Phrase 3 45

sankarAbharanA Phrase 1 81Phrase 2 51Phrase 3 101

varALi Phrase 1 52

Table : Phrases for ModellingSlide 7/21


Motif from Signal Procesing Perspective

• Motifs can be thought of as signature prosodic phrases.• Different rAgas may be composed of the same set of notes, or even

phrases, but the prosody may be completely different.• Prosodic modifications include :

• Increasing/decreasing the duration of notes.• Using an appropriate intonation pattern by employing gamakas.• Modulating the energy.

• Thus a Motif is a ”Prosodic Phrasing of a Sequence of Notes”.

Slide 8/21


Pitch Contours of Motifs Labelled in bhairavI and sankarAbharanA

0 50 100 150 200 250 300100

200

300Bhairavi Motif 1

0 200 400 600 800 1000 1200100

200

300Bhairavi Motif 2

0 200 400 600 800 1000 1200100

200

300

400Shankarabharanam Motif 1

100 150 200 250 300 350 400 450 500100

200

300


0 50 100 150 200 250 300 350 400 450 500100

200

300


Figure : Motifs of bhairavI, sankarAbharanA

Slide 9/21


Pitch Contours of Motifs Labelled in kAmbOji, kalyAni and varALi

0 50 100 150 200 250

200

300

400Kamboji Motif 1

0 50 100 150 200 250 300 350100

200

300

400Kamboji Motif 2

0 50 100 150 200 250 300 350

200

300

400Kamboji Motif 3

0 50 100 150 200 250 300 350

200

300

400Kalyani Motif 1

0 50 100 150 200 250 300 3500

200

400Varali Motif 1

Ri Ga

Figure : Motifs of kAmbOji, kalyAni and varALi

Slide 10/21


DYNAMIC TIME WARPING

• DTW gives almost accurate for continuous feature vectors.• DTW also gives good distance measures if the query and reference are

similar looking.

0 20 40 60 80 100 120 1400

50

100

150

200Mapping

70080090010001100120013000

50

100

150

200Reference

0 20 40 60 80 100 120 140700

800

900

1000

1100

1200

1300Query

0 50 100 150 200700

800

900

1000

1100

1200

1300Warped Query with Reference

Figure : DTW

Slide 11/21


DTW ON PITCH CONTOUR

• Various musicians sing the same phrases with substantial variations.• This results in dissimilarities in reference and query vectors.• Feature Vectors for the phrases are discontinuous in many cases.• Hence DTW gives distorted distance measures for such cases.

0 20 40 60 80 100 120 140 160 180 200600

800

1000

1200

1400Reference

0 20 40 60 80 100 120 140 160 180 200600

800

1000

1200

1400Query

Figure : Dissimilar Contours

Slide 12/21


60070080090010001100120013000

20

40

60

80

100

120

140

160

180

200

0 50 100 150600

700

800

900

1000

1100

1200

1300

0 50 100 1500

20

40

60

80

100

120

140

160

180

200

0 50 100 150 200600

700

800

900

1000

1100

1200

1300

Figure : Distorted DTW measure

Slide 13/21


Markov Model for Music

o o o o o o1 2 3 4 5 6

b (o ) b (o ) b (o ) b (o ) b (o ) b (o )1 2 3 4 5 6

a a

a a

a a

12

11

13

23 34

24

44

1 2 3 4a

a33

a22

ObservationSequence

1 1 2 3 3 4

a = 0ij

, j < i π = 0, ι ιιπ = 1, ι = 1

= 1

Slide 14/21


Building of Models - Training

Slide 15/21


Testing

P(O|

P(O|

)

P(O| )1

)2

v

Sequence

Pitch Contour

FeatureAnalysis,(Pitch

MusicSignalS Extraction)

HMM forMotif 2

Select

Index ofRecog.Motif

Observation

λ

λ

HMM forλ

λ

λ

λ

1

2

v Motif v

ComputationProbability

ProbabilityComputation

ProbabilityComputation

Maximum

Motif 1HMM for

Slide 16/21


CONTINUOUS DENSITY HMM’S FOR MODELING MOTIFS• HMM’s were chosen for Modenling Motifs to accommodate Prosody of the

Motif.• Number of notes in the Motif determined the HMM Structure.• The Number of States determined based on the changes obsereved in the pitch

contour.• Two Mixtures used for each State.

600

800

1000

1200

1400

600

800

1000

1200

1400

State Occupancy on Viterbi Alignment

600

800

1000

1200

1400

600

800

1000

1200

1400

5

11

11 12

12

82 3 5

2 4 8 11 12

98532

12852

3

3 11

Figure : HMM AlignmentSlide 17/21


MOTIF RECOGNITION RESULTS• A total of 10 motifs were experimented with.

• 2 Motifs for Bhairavi• 3 Motifs for Shankarabharanam• 3 Motifs for Kamboji• 1 Motif for Kalyani• 1 Motif for Varali

• The table below gives the confusion matrix for Motif Recognition

rAga bh1 bh2 ky1 kb1 kb2 kb3 sk1 sk2 sk3 va1bh1 40 2 1 0 0 6 0 0 0 3bh2 0 61 4 1 4 1 1 0 0 0ky1 0 1 23 10 0 0 11 2 5 0kb1 0 0 3 91 0 0 6 3 1 0kb2 0 0 1 2 44 0 0 0 1 0kb3 0 0 2 1 0 41 0 0 0 0sk1 0 2 18 13 3 0 28 7 9 0sk2 0 0 7 0 1 0 4 34 2 4sk3 0 1 23 25 1 0 29 5 10 2va1 3 0 1 0 0 0 0 0 0 48

Table : Confusion matrix for motif recognition using HMMs(bh – bhairavi, ky –Kalyani, kb – Kamboji, sk – Sankarabharana, va – Varali)

Slide 18/21


PHRASE SPOTTING

• J.Rose et al. illustrate a method to spot motifs in a Bandish usingpercussive cues from the Tabla. 1

• The phrase most probably lies in one intonation unit or breath group.• AlApanAs have to be split into these intonation units or breath groups.

1J.Rose and P. Rao, Detecting melodic motifs from audio for hindustani classic music, in International Society for Music Information

Retrieval Conference, Portugal, October 2012.

Slide 19/21


References

• S. A. H G Ranjani and T. V. Sreenivas, Shadja, swara identification and rAgaverification in alapana using stochastic models, In 2011 IEEE Workshop onApplications of Signal Processing to Audio and Acoustics (WASPAA), pp.2932, 2011.

• J.Rose and P. Rao, Detecting melodic motifs from audio for hindustani classicmusic, in International Society for Music Information Retrieval Conference,Portugal, October 2012.

• P. Rao, Audio meta data extraction: The case for hindustani music, in SPCOM,Bangalore, India, July 2012.

• M. Subramanian, Carnatic rAgam thodi pitch analysis of notes and gamakams,Journal of the Sangeet Natak Akademi, vol. XLI, no. 1, pp. 328, 2007.[Online]. Available: http://carnatic2000.tripod.com/thodigamakam.pdf

• R. Sridhar and T. Geetha, rAga identification of carnatic music for musicinformation retrieval, International Journal of Recent trends in Engineering, vol.1, no. 1, pp. 14, 2009. [Online]. Available:http://ijrte.academypublisher.com/vol01/no01/ijrte0101571574.pdf

Slide 20/21


References• P. Chordia and A. Rae, Raag recognition using pitch-class and pitch-class dyad

distributions. sterreichische Computer Gesellschaft, 2007, pp.431436. [Online].Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.1557

• G. Pandey, C. Mishra, and P. Ipe, Tansen: A system for automatic rAgaidentification. Citeseer, 2003, p. 13501363. [Online]. Available:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.60.8712

• A. Krishna, P. Rajkumar, K. Saishankar, and M. John, Identification of carnaticraagas using hidden markov models, in Applied Machine Intelligence andInformatics (SAMI), 2011 IEEE 9th International Symposium on, jan. 2011,pp. 107 110.

• S. Shetty, rAga mining of indian music by extracting arohana-avarohanapattern, International Journal of Recent trends in Engineering, vol. 1, no. 1,2009. [Online]. Available:http://ijrte.academypublisher.com/vol01/no01/ijrte0101362366.pdf

• D. Swathi, Analysis of carnatic music: A signal processing perspective, MSThesis, IIT Madras, India, 2009.

• A. Bellur, V. Ishwar, and H. A. Murthy: A knowledge based signal processingapproach to tonic identification in indian classical music, in Workshop onComputer Music, Instanbul, Turkey, July 2012.

Slide 21/21

compmusic workshop ii: motivic analysis and its relevance...

Documents