The evaluation and optimisation of multiresolution FFT ParametersFor use in automatic music transcription algorithms

Automatic music transcription (AMT)

AMT Algorithms

Time & Frequency Resolution

Time Resolution IncreasesFrequency Resolution Decreases

Short Window

Time Resolution DecreasesFrequency Resolution Increases

Long Window

Multiresolution FFT (MRFFT)

High FrequencyResolution

High Time Resolution

FcA FcB FcC FcD

FFT A FFT B FFT C FFT D

Time Freq Plane - Dressler

Window Length - Bin Alignment

Note-bin alignment – The position of a fundamental frequency relative to a FFT bin frequency.

Note bin alignment

215.33236.87258.40279.93301.46323.00344.53366.06387.60409.13430.66452.20473.73495.260

50

100

150

200

250

A 2048 FFT Decomposition of a 376.83Hz Sine Wave

FFT Bin (Hz)

FFT B

in M

agnit

ude

Note bin alignment

215.33236.87258.40279.93301.46323.00344.53366.06387.60409.13430.66452.20473.73495.260

100

200

300

400

500

600

A 2048 FFT Decomposition of a 366.06Hz Sine Wave

FFT Bin (Hz)

FFT B

in M

agnit

ude

MRFFT Optimisation

Cut off frequencies

Subband FFT Length

Optimised based on 3 characteristics determined by window length Time Resolution Frequency Resolution Note Bin Alignment

Scoring

Calculate score for time, freq, and note-bin alignment in each subband

Weight score according to notes in subband

Range correct score to be between 0 and 1

Sum all scores across all bands to generate MRFFT Score

Note Bin Scoring

If 2 note frequencies fall within same bin, FFT length is discounted as unsuitable

Weighted Sub-band FFT Bin Score = Sub-band FFT Bin Score * (notes in sub-band/total notes across all bands)

Scoring Process

The algorithm moves the cut off frequencies A, B and C through all combinations of positions. For each position, all FFT lengths between 256 and 8192 samples in increments of 128 are evaluated on each sub-band. All combinations of FFT lengths on all combinations of subbands are evaluated and scored.

Subband A Subband B Subband C Subband D

FcA FcB FcC FcD80 Hz 5KHz

Solutions

1. 4 band MRFFT 256-8192 range

2. 3 band MRFFT256-8192 range

3. Dressler 4 band MRFFT256-2048 range

4. Dressler fixed FFT Length variable bands 256-2048 range5. 4 band MRFFT

256-2048 range6. 1 band FFT

8192

Resu

lts

– S

ub

band

Div

isio

ns

Band A

Band B

Band C

Band D

Results – MRFFT Score

Transcription Test – Low F Bands

FcA FcB

Original

Solution 1

Solution 6

High F Resolution of solution 6 is reflected inLow frequency transcription accuracy

Transcription Test – High F Bands

Solution 1

Solution 3

Solution 6

F-Measure Results

1 2 3 4 5 60.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

RecallPrecisionFmeasure

Solution

Score

Recall refers to the fraction of the relevant notes that were retrieved i.e. how many of the correct notes the system extracted.

Precision refers to the fraction of relevant notes retrieved, relative to the total number retrieved. I.e. how many of the extracted notes that were correct.

F-Measure is the weighted mean of precision and recall.

Peak Picker

A threshold is dynamically set for each analysis window of the STFT as a percentage of the maximum magnitude within the window, with a minimum threshold heuristically decided. If a bin magnitude exceeds the threshold a note is transcribed at that point.

Peak Picker Robustness

Solution 1 Vs Solution 6 Picker

MRFFT Implementation

6016 FFT is performed on the entire frequency spectrum. The spectral information is then filtered to include only the frequencies required by that band.

note frequency (orange magnitude) not in the frequency band considered, generates cross channel interference (red magnitudes) that contributes to the magnitudes in the sub-band of interest.

Cross talk indicators

Adjacent bins Adjacent bins in optimised MRFFT

represent fundamental frequencies. Therefore any cross channel interference will contribute to energy contained in FFT bins representing note frequencies. This may contribute to false positives.

F Measure conclusions

The results of the F-Measure are largely disappointing, and can be attributed to the inadequacies of the implemented peak picker to handle fluctuations in magnitude of local maxima. Characteristics of the MRFFT, like adjacent note representing bins, and interference generated by sub-band division methods contribute to this problem.

Large variations of spectral magnitudes also contribute

Conclusions

The theoretical scoring of MRFFT parameters resulted in favourable results for the optimised FFT.

The ‘real world’ sinusoidal extraction test demonstrated initially disappointing F-Measure results for the MRFFT solutions compared to the single band 8192 FFT. However, upon closer analysis of the transcribed files, positive aspects of the MRFFT analysis were found as performance improved in the higher frequencies.

Further investigation of the results revealed inadequacies of the peak picker implemented and also indicated issues with the construction of the MRFFT that require further investigation.