trackingtrackingbeatbeat

trackingtrackingbeatbeat

Mcgill university :: music technology :: mumt 611>>

trackingtrackingbeatbeat

… WHAT IS BEAT TRACKING?

/170

trackingtrackingbeatbeat

What is beat tracking?

… magic box …… magic box …

Input audio

output tatum locations

/171

trackingtrackingbeatbeat

What is beat tracking?

“…Estima(tion) of the possibly time-varying tempo and locations of each beat. In Engineering

terms, this is the frequency and Phase of a time-varying signal, the phase Of Which

is zero at a beat location (I.e., where One would tap one’s foot).”

Hainsworth, 2006

/172

trackingtrackingbeatbeat

Overview

Motivation & definitions …

early work …

Discrete vs. continuous detection functions …

canonical work: scheirer …

autocorrelation versus comb filtering …

Exemplar Methods …

Persistent problems …>>

>>

>>

>>

>>

>>

>>

/173

trackingtrackingbeatbeat

Motivation & definitionsAuto accompaniment

Synchronization of 2 streams

Cd skipping recovery

Time-scaling algorithms

Tempo-synchronous effects/control

Database retrieval

similarity

>>

>>

>>

>>

>>

>>

>>

/174

trackingtrackingbeatbeat

Motivation & definitionsBlimes divisions of musical timing… metrical structure… tempo variation… timing deviations… arrhythmic sections

3 hierarchal levels of metrical structure… Tempo… tactus… tatum

Beat tracking / Tempo induction

>>

>>

>>/175

trackingtrackingbeatbeat

Approach overview

Rule based… steedman (1977)… parncutt (1994)

Autocorrelation … Brown (1993)… *Davies & Plumbley (2005)

Oscillating filters… *Large (1994)… *Scheirer (1998)

histogramming… *seppanen (2001)

Multiple agent… *goto (1995)… Dixon (2001)

probabilistic… hainsworth & macleod (2003)… *klapuri (2003)

>>

>>

>>

>>

>>

>>

Red audioBlack symbolic* causal

==

= /176

trackingtrackingbeatbeat

Early Work

>>

>>

>> music perception and comp sci (1980’s)

Most early work with midi/symbolic data

Rule based

/177

trackingtrackingbeatbeat

Early Work

steedman (1977)

large (1994)

Goto (1995)

Scheirer (1998)

>>

>>

>>

>>

>> MIDI

AUDIO

.

.

.

/178

trackingtrackingbeatbeat

Discrete df vs. continuous df

Discrete detection function

… localized onset points, or IOI (inter-onset intervals)

… Suited for monophonic signals

… step 1: Created by various comparative time or time-freq techniques

… step 2: peak picking technique

Continuous detection function

… better for unknown onset densities

… same as step 1 above

… further processing required for important results

>>

>>

/179

trackingtrackingbeatbeat

Scheirer

Input audio

Filt_1Filt_2Filt_3Filt_4Filt_5Filt_6

Env_1

frequencyfilterbank

Continuousenveloping

...

Combfilterbank

. . .

. . .

. . .

. . .

. . . ...

...

Sumfltbks

Peakpick

/1710

trackingtrackingbeatbeat

ACF vs Comb filt

Automatic phase alignment

Possible tempi at multi & fracs

Meter estimation directly avail

Phase Alignment in 2nd step

Commonality not given directly

Meter estimation via decim. & sum

Less expensive

>>

>>

>>

>>

.Comb filters*

>> >>

>>

/1711

trackingtrackingbeatbeat

goto

Input audio Dscrt

Onsetdet

Acf

period

Prior knowledge: 1) frequent ioi is likely ibi 2) sounds likely to occur on beats 3) rhythmic pattern templates 4) chord templates for non-perc music

Priorkn0w

Sub_1Sub_2Sub_3

Sub_7

freqfltrbk

...Crosscorr

alignMulti agents

winner

/1712

trackingtrackingbeatbeat

klapuri

Input audio

Filt_01Filt_02Filt_03

Filt_36

freqfltrbk

...

Chan_1

Chan_2

Chan_3

Chan_4

...

Combfltrbk

...

...norm

Pwr envPeriod &

Align estim Bar

beat

tatum

Sn-1

Bar

beat

tatum

Sn

HMM

P(sn qn)| Observable variable conditioned by current state=

Traindata

Training data rhythmic pattern templates=

/1713

trackingtrackingbeatbeat

Davies & plumbley

Input audio

ContDetectfunc

acf

periodicity alignment

Contdep

stateGenstate

2 state model

...

Combfltrbk

...

...

...

Combfltrbk

...

...

/1714

trackingtrackingbeatbeat

comparison

Davies & plumbley

klapuri

scheirer

55.9 61.4 71.2 80.9

23.8 38.9 29.8 48.5

54.8 61.2 68.1 78.9

raw allowedCml% Tot% Cml% Tot%

Raw Cml correct metrical level, continuity requiredraw tot correct metrical level, continuity not requiredAllowed cml 1/2 & 2x tempo allowed, continuity requiredAllowed cml 1/2 & 2x tempo allowed, continuity not required

==

==

/1715

trackingtrackingbeatbeat

Persistent problems

Periodicity switching

Half/double time

Alignment issues

Expressive timing

Non-percussive music

Areas for future work

>>

>>

>>

>>

>>

/1716

trackingtrackingbeatbeat

conclusions

much progress has been made through several approaches

Possible New methods of extracting periodicity and phase

we need to work on improving the robustness of calculations

Thank you for your time!

>>

>>

>>

/1717

trackingtrackingbeatbeat

referencesDavies, M.E.P., M. Plumbley. “Context-dependent beat tracking of musical Audio,” IEEE Transactions on Audio, Speech and Language Processing, 15(3), 2007, pp. 1009-20.

Goto, M. “A study of real-time beat tracking for musical audio signals.” PhD thesis, waseda university, 1998.

Hainsworth, s.w. “beat tracking and musical metre analysis,” in Signal processing methods for music transcription, edited by a. Klapuri, and M. Davy, 101-129.New york: Springer science and business media, 2006

Hainsworth, s.w. “techniques for the automated analysis of musical audio”,PhD thesis, department of engineering, university of cambridge, 2004.

KLAPURI, A. “SIGNAL PROCESSING METHODS FOR THE AUTOMATIC TRANSCRIPTION OF MUSIC” PHD THESIS, TAMPERE UNIVERSITY OF TECHNOLOGY, 2004.

Scheirer, e. “music listening systems”, PhD thesis Massachusetts institute ofTechnology, 2000.