Lyric alignment in popular songs

Luong Minh ThangWING group meeting 12 Oct, 2007

Outline

RecapRepetitionGlobal alignmentLocal alignmentSong model & Segmentation

Recap

Given: textual transcription of lyrics

acoustic musical signal of a song

Purpose: find the timestamps for the beginning and ending points for each line of the song.

Recap (cont.)

i

j

Lyric self-similarity matrix

Chroma self-similarity matrix

Lyric-chroma similarity matrix

i

jk

kja ]][[

How to generate lyric-chroma similarity matrix?

Repetition

self-similarity matrix

j

k

kja ]][[

vj

Linear sequence: v1, v2, .., vm

+ 2 units I, j will have vi, vj approximately equal

+ count how many times vi are repeated

+ count how many times vi vi+1 ..vi+k are repeated

+ average across length

Repetition sequence: r1, r2, .., rm

Repetition (cont.)

Chroma repetition contains noise Need to be improved here

Local alignment

Use non-overlapped local alignment

Chorus 1

Chorus 2

Chorus 3

Chorus 4

Chorus 2 vs. Chorus 1

Global alignment

Scoring matrix to restrict skip on lyric side Free-end versions

Local alignment (cont.)

Looking at a specific windowUse local alignment to detect other

segments

Some figures …

Song model

Divide dataset into 2 categories: I-V1-C1-V2-C2-O & I-V1-C1-v2-C2-V3-C3-O

Model for each category to tell the estimated positions of a particular

Segmentation

Align the same segments in lyric and chroma together.

Use perfect knowledge to set an aim

Some figures …

To be done

Incorporate segmentation on chroma side

Continue writing thesisImproving repetition values

Thank you !