a high resolution fundamental frequency determination based on phase changes of the ...

7/30/2019 A high resolution fundamental frequency determination based on phase changes of the Fourier transform

1/6

A high resolution fundamental frequency determination basedon phase changes of the Fourier transformJudith C. BrownPhysics epartment, WellesleyCollege,We/lesley,Massachusetts 12$1andMedia Lab, MassachusettsInstituteof Technology, ambridge,Massachusetts2139Miller S. PucketteIRC4M, 31 rue St Merri, Paris 75004, France(Received16 June 1992; evised April 1993;accepted April 1993)The constantQ transform described ecently [J. C. Brown and M. S. Puckette, "An efficientalgorithm or the calculationof a constantQ transform,"J. Acoust.Soc. Am. 92, 2698-2701(1992}] has been adaptedso that it is suitable or tracking the fundamental requencyofextremely apid musicalpassages.or this purpose he calculationdescribed reviouslyhasbeenmodifiedso that it is constant requency esolution ather than constantQ for lower frequencybins.This modifiedcalculation erves s the input for a fundamental requency racker similarto that described y Brown [J. C. Brown, "Musical fundamental requency racking usingapattern ecognitionmethod,"J. Acoust.Soe.Am. 92, 1394-1402 ( 1992)]. Once he fastFouriertransform FFT) bin correspondingo the fundamentalrequencys chosen y the frequencytracker,an approximations used or the phasechange n the FFT for a time advance f onesample o obtainan extremelyprecise alue or this requency.Graphicalexamples re given ormusicalpassages y a violin executingvibrato and glissandowhere the fundamental requencychanges re rapid and continuous.PACS numbers: 43.75.Yy, 43.60.Gk

INTRODUCTIONOur fundamental requency racker s basedupon thecalculationof a constantQ spectral ransformdescribedrecently by Brown and Puckette (1992). Here we showed

that the direct calculationof the components f a constantQ transformcan be accomplishedmore efficientlyby atransformation of the fast Fourier transform (FFT). Tosummarize that calculation, the direct calculation can becarried out using

v[k-q[kq] w[n,kcq]x[e-J'k,,where q[kcq]s the keq omponentf the constanttransform,o[n,kcqis a windowunctionf length [kcq,x[n]sasampledunctionf ime,nd okqs herequencyof this component.This can be evaluated using the following form ofParseval's quation

N- I v- x[n]*[n,kc]----=oif we define

w[n,ko]e-Okt,"=.7F[n,kcqlSincehe unctions/*[n,kq]have ery ewnonzeroom-ponents, he fast Fourier transformX[k] of the signalx[n]can be transformed nto a constant Q transform with veryfew additional operations.For application to the performanceof modem com-puter music, which includesextremely rapid passages,

time resolution of 25 ms or less is desirable. We are thusconfrontedwith the usualdilemma n choosing n accept-able trade-offbetween emporal and frequency esolution.Our compromise onsists f limiting the temporalex-tent of the window.This means hat the low-frequency insof our transformaxeconstant requency esolution equalto the sample rate over the temporal window length)rather than constantQ. For example,we may choose orthe center frequencies f the bins of the transform o cor-respond o the frequencies f notesof the equal temperedmusicalscalebeginningwith the first bin at C3 ( 130.9Hz).Then a window engthof 25 ms means hat the resolutionis a constant qual o 40 Hz (while the Q is variable)up toa frequencyof 717.8 Hz or the 30th bin. The resolution sthen variable with Q constantequal roughly 17 up to afrequencyof 5274 Hz or the 65th bin. We will call thistransform the modified constantQ transform, and we willshow that limiting the temporal extent of the window forthe low-frequencybins doesnot lead to decreased erfor-mance or the detectionof the fundamental requency. tdoes, of course, mean greater spillover for the low-frequencybins.An exampleof this calculation an be found n Fig. 1for a clarinet playing a chromaticscale where we haveplotted the amplitudeof the modifiedconstantQ transformcomponents gainstbin number in each frame. Time isincreasingvertically for these frames. Here the lower Q's(with the lowest value about 5) of the low-frequencybinsare manifestedby a greater bandwidth. This figure can becompared o Fig. 4 of Brown and Puckette (1992) for aspectrumof the samesoundcalculatedwith a "true" con-

662 J. Acoust.Soc. Am. 94 (2), Pt. 1, August1993 0001-4966/93/94(2)/662/6/$6.00 @ 1993 Acoustical ocietyof America 662


2/6

CLARINET SCALE VIOLIN SCALE

121

32.5 .cFFT BIN NUMBER -> 65 o

80o

700

A 600

300

200

0 I 2 2.53time (s) ->

FIG. 2. Frequencyracking esults or a violin playing wo octaves f adiatonicscale.The opensquares the result rom the modified onstantQfundamentalrequencyracker.The solid riangle s the preciserequencyreturnedby the high-resolution hasecalculation.FIG. 1. Amplitudeof the modified onstantQ spectral omponents lot-ted against in number correspondingo frequency) or a series f timeframes.A clarinetwith microphone lacedn the barrel s playing everaloctaves of a chromatic scale.

stant Q of 17. It is interestingo note that the evenhar-monics re missing s predictedor a clarinetup to aboutthe 29th bin and at this point the energy n the third har-monicbegins o decrease. he clarinetsoundwas recordedwith the microphonen the barrel of the instrument o thespectrum s not as rich as for a normal clarinet sound.I. CALCULATION OF INITIAL FREQUENCY ESTIMATE

When the constantQ transformof a soundconsistingof harmonic requency omponentss plottedagainst ogfrequency, he spacingof thesecomponentss invariant(Brown, 1991). The fundamental requencycan then bedeterminedby finding he position n log frequency paceof this invariantpattern.This is bestaccomplishedy cal-culating he cross-correlationunctionof each rameof thelog frequency pectrumwith the ideal patternas discussedby Brown (1992).The "ideal pattern" used in calculating he cross-correlation unctionconsists f components ith the fre-quencyspacingdiscussed boveand with amplitudesde-creasing inearly from 1 for the fundamental o 0.6 for thehighestharmonic.The purpose f varying he amplitudesis to prevent he choiceof the frequency n octavebelowthat of the true fundamental.For this positionof the idealpattern, all even components f the ideal pattern line upwith components resent n the spectrum. his is becausethe spacing f components f, 4f, 6f, etc. is the sameasthat of components,, 2f, 3f, etc. If all componentsreweightedequally, the value of the cross-correlationunc-

tion will have he samevalue or evencomponentsf thepatternalignedwith the componentsf the signalas or the"true" positionwhereall componentsf the pattern all on

their counterparts n the test spectrum. However, withsmaller weights on higher components, his error isavoided.We haveestimated he run time for our algorithmwithcalculations arried out on a 40-MHz Intel i860 usingahand-coded outine. With a 512-point FFT and quartertonespacing ver hreeoctaves,he FFT takes343/rs andthe transform1664-2/s (measured n an oscilloscope).The cross-correlation calculation involves well under 250

multiplies venwith ten componentsn the harmonicpat-tern so the computationime for this operations negligi-ble. The overallcomputation ime then is on the order of0.5 ms. This can be compared o a time advance f 25 msbetween rames, so the calculation s easily carried out inreal time.

Resultsof our calculationapplied o digitizedviolinand clarinet scales re found in Figs. 2 and 3 where theopensquares orrespond o the frequencies f the notesofthe equal tempered calechosenby our frequency rackerplotted against rame number.The solid triangleswill beexplainedn the next section.Each framecorrespondso atime advance f 25 ms n the sound.Theseare examples finstrumentswith very differentspectra.The violin has acomplex spectrumwith many higher harmonicspresent.This clarinet soundwas recordedwith the microphone nthe barrel and has a relatively simple spectrumwith asmallernumberof harmonics. he numberof componentsin the ideal pattern used for the crosscorrelationvariedaccordingly,with 10 componentsor the violin and threecomponents or the clarinet. There were essentiallyno er-rors in determining undamental requencies f the notespresentwith our modifiedconstantQ transform.II. HIGH-RESOLUTION FREQUENCY DETERMINATIONA. Phase background

The importanceof the phase n the discreteFouriertransform DFT) has beendiscussedy Oppenheim nd663 J. Acoust. oc.Am.,Vol.94, No. 2, Pt. 1, August 993 J. Brown nd M. PuckeRe: igh esolutionrequency etermination 663


3/6

2OOO

IOO0

CLARINET SCALE

time (s) ->

FIG. 3. Frequencyracking esults or a clarinetplayingseveral ctaveso a chromaticcale. heopen quares theresultrom hemodifiedconstantQ fundamental requency racker.The solid riangle s the pre-eisa requencyeturned y the high-resolutionhase alculation.Luis ( 1981 . They pointout that in a varietyof disciplinesphase nly nformationeads o a morerecognizableecon-structionof the originalobjectanalyzed han does nfor-mation basedon magnitudeonly. Friedman (1985) dem-onstrated that one can obtain narrower formant bands forsonagrams f speechusing a histogramof occurrencesagainst requencywith the frequencyobtained rom thetime derivative of the phase of the short-time Fouriertransform.Computationallyhis wasobtained y calculat-ing two short-time ourier transform STFT's) with thederivative of the window function used in one of them.

Earlier, in work on the phase roeoder applied tospeech ignalsFlanaganand Golden (1966) usedphasedifferenceso obtain greateraccuracyof Fourier compo-nents. Beauchamp 1966, 1969) and Grey and Moorer(1977) useda similar echnique pplied o musical ignals.See also Moorer (1978) and Dolson (1986).Charpentiex1986) described pitch trackerbasedonfrequenciesbtained rom an approximationor the phasedifferenceof time frames of the STFT separatedby onesample.We obtained his sameexpressionndependentlyand will discusst in the followingsection.B. Phase calculation

The method of frequencydeterminationwhich wehave described in the Introduction and in Brown (1992)works extremely well for instruments playing discretenotes belonging o the equal tempered scale. Here thesmallest requencydifferencebetweennotes s approxi-mately 6%, and the resultsare reportedas notesof theequal empered cale.However,a very differentsituationcan arise n passages layedby stringed nstruments rwind instruments. These instruments are not constrainedto play discretenotesas are keyboard nstruments. husthe frequency an vary continuouslys n, for example,glissando r vibrato.Keyboard nstrumentsmay alsobe

tuned o temperamentsther han equal empered. or allof thesecases he frequencydeterminationmust be muchmore accurate than 6% in order to track the audio wave-form.

The frequencyof a particular Fourier component sobtained rom the bin into which it falls in the magnitudespectrum s only as accurate as the frequencydifferencebetweenbins, in our case 6% or 3% dependingon thecalculation. his estimate an be improvedby a quadraticfit using he amplitudes f the bin containinghe maximumand the two adjacentbins and identifying the positionofthe maximum of the parabola hus obtained.This is anextremely well-known technique described recently bySmith and Serra (1987). We will discuss he accuracyofthis approximationn a later section.Even more accurate s a method we have developedbasedon an approximation or the phasechangeper unitsample or the Fourier component hosenas the correctfundamental requency y our frequency racker. t is wellknown that the frequency s determined y the change nphases muchmoreaccuratehan that obtainedrom themagnitudespectrum.However, the problem with deter-mining he requencyrom hephase ifferencevera reh-sonable op size (samples etweenrames) s oneof phaseunwrapping. he phasechange s only known modulo2z.This problemdoesnot arisewith a hop sizeof one samplesince he highestdigital frequencys r radians/sample, utthis case necessitateshe computationof an additionalFFT.

In fact this additionalcomputation an be avoidedbyusingan approximation hichassumeshat x[n] is peri-odic.The phase hangeor a hopsizeof onesample anbeobtained from the following identity (Oppenheim andSchafer,1975; Charpentier,1986). If -{x[n]}=X[k] isthe kth component f the discreteFourier transformofx[n], then

-{x[n+m] }_eJ2'km/VX[k] ( 1is the DFT after m samples.The above quation pplies o an unwindowed FT. Itis possibleo use his result o obtaina Hanning-windowedtransform since he effectof windowingcan be calculatedin the frequency omain or this window.We will use henotation Zt[k]o denoteheHanning-windowedouriertransform evaluated or a window beginningon sampleno,that is

Xu[ t,no]=where

N--I x[n+no]w[n]e-J2'kn/

i 1w[ n ] =-- cos(2rn/N)-- 1/2 1 (!lj2;n/N l"--j2rn/N1

Substituting his expressionor the window n the pre-cedingequation eads oXH[k,no] {X[k]---}X[k+ 1 ---}X[k-- 1 }. (2)

664 J. Acoust.Soc. Am.,VoL 94, No. 2, Pt. 1, August 993 J. Brownand M. PuckeRe: ighresolutionrequency etermination 664


4/6

Using Eq. (1) with rn= 1 in Eq. (2), the approxima-tion for the Hanning-windowed FT after one samplesX[ k,no+1 ={eJ2k/NX[-eJ2(k /NX[ + 1]

_ e22(k-)/vX[_ l ]}.And finallyX[ k,no+I =eJ2/V{X[--ed2r/vX[+ 11

--e-J2/VX[--1] (3)The digital requencyn radians er sampleor the kthbin correspondingo the phasedifference or a time ad-vanceof one sample sto(k,no) = (k,n0+ 1 -qb(k,no), (4)

where[Im(Xr[k,no+1 ) k,no1=arctanReXHk,no1

and flm(XH[k,no]4(k,n0)arctanRe(a[,no.This expressionor the phasedifference olds or any DFTbin with the bin indicatedby k. For usewith a fundamentalfrequency racker, the calculationwould only be usedonthe bin selected s winner by the tracker.III. RESULTS

To check his methoda test ile was generatedn soft-ware consisting f the superposition f sinusoidal ompo-nents of equal amplitudeat frequencies 34.97, 1739.88,and 3479.77Hz. These requencies erechoseno fall intobin positions10.1, 40.4, and 80.8. A 256-pointDFT wastaken, and the real and imaginarycomponents btainedwere substitutednto Eq. (4) using he definitions ollow-ing this equation.The calculationwas carriedout for eachof the 128 positive requency omponents f the DFT.The result is found in Fig. 4 where we have plottedto(k) (converted o Hertz for a samplerate of 11025)againstbin number. Note that the calculated requenciesare correct for five or more bins on either side of the bininto which the frequencybelongs.This point will be dis-cussed urther in the sectioncomparing his method withthe phasevocoder.The measured requencies s determinedby Eq. (4)were also printed out and were correct o the two decimalplaces ndicatedabove.The solid diagonal ine representsthe center requency f theseDFT binsplottedagainstbinnumber.The analogous raphobtained rom an exactmea-surement of to based on the calculation of two successiveDFT's with a time advanceof one samplewas dentical othis graph and is not included. This means that the as-sumption of periodicity used in Eq. (1) is extremely goodfor a hop size of 1 sample.

When used as the back end of a fundamental fre-quency racker, the determinationof the precise requency

TEST FILE WITH THREE SINUSOIDS5000

3000

20001000

0 100FFT BIN NUMBER ->

FIG. 4. High-resolutionrequencyscalculatedn Eq. (4) fi3reachbinplottedagainstbin number or a test file with threecomponents.

as described ddsa negligible mount o the computationtime since t is only carriedout for oneDFT bin. Once hebin number for the initial estimate of the fundamental fre-quencys selectedrom the constantQ transform, calcu-lation s made o determine he correspondingin numberfor the FFT. The real and imaginarypartsof the FFT forthis bin and thoseon either sideof it were previously al-culated,and only these hree complexnumbersare neededfor the evaluationof Eqs. (2) and (3). Theseare then usedin Eq. (4) with the definitionsollowing t.The precise requencies ere calculatedor the violinand clarinet scalesdiscussed reviously.The resultsaregivenby the closedrianglesn Figs.2 and 3. Note that forthe violin the precise requencys correct n severalcaseswhere he initial estimatewas off by a bin. For the clarinetin Fig. 3, the preciserequencys correct or the first hreenotes where the initial estimate was incorrect because theconstantQ transformdid not extend o that low a fre-quency.The powerof this method s evenmoreapparentwhenit is applied o the acoustic oundsor which t is :intended.In Figs.5 and 6 are found he frequenciessolid riangles)obtainedusingEq. (4) on the output from the frequencytracker describedn Sec. . A violin is executing glissandoin Fig. 5 and vibrato n Fig. 6. For comparison e includethe opensquaresndicating he results or the frequencytracker without the phasecorrection.For the glissandotwo pointson the preciserequency urveare slightlyoff,but these errors are small. We are uncertain as to theirorigin.IV. COMPARISON OF ACCURACY OF PHASEMETHOD WITH QUADRATIC FIT

For a sampled function y(x) where values are onlyknown for integralx, the positionof the "true" maximumof the function usually occursat nonintegralvaluesof x.One widely used meansof approximating he (noninte-665 J. Acoust.Soc. Am., Vol. 94, No. 2, Pt. 1, August1993 J. Brownand M. Puckette:High resolution requencydetermination 665


5/6

900

800

700

600

500

VIOLIN GLISSANDO

v vv

time (s) -

TABLE I. Error in quadratic it calculation.The first column is theposition f the true maximumof the function elative o zero.The secondcolumn s the positionof the maximumpredicted y the quadratic it inEq. (6), and the third column s the error in the quadratic it calculation(difference in columns one and two).

r Q fit Error--0.500 --0.500 0.000--0.400 --0.357 --0.043--0.300 -0.247 --0.053--0.200 --0.156 --0.044-- 0.100 -- 0.076 -- 0.024

0.000 0.000 0.0000.100 0.076 0.0240.200 0.156 0.0440.300 0.247 0.0530.400 0.357 0.0430.500 0.500 0.000

FIG. 5. High-resolution requencyplotted against ime for a violin exe-cutinga glissando. pensquaresepresenthe resultsof the fundamentalfrequencyrackerwith solid riangleshe high-resolutionesults.gral) x valuewhere he true maximumoccurs s by fittinga parabola hrough he point where the maximumof thesampled unctionoccursand the pointson either side.If the maximumoccurs t x = 0 andy (0) =Y0, then thethreepoints -1,y_), (0,y0),and (1,y+) labeledwiththeir x and y valuesare assumedo lie on a parabola.Witha little algebra t is easy o show hat the maximum of theparabolaoccursat

Y+t--Y-x= (5)2(2yo-y+t-y_) 'The accuracy or the quadratic it methodcan be cal-culated exactly for an input signalconsisting f a singleharmonic omponent. or a componentalling nto bin k0with a frequencycorresponding xactly to ko+r where-0.5


6/6

TABLE II. Comparisonf averagerequenciesredicted y phase if-ference pproximationrow two) andquadraticit (row three) with theirstandard eviationsrom 10 measurements.ow onegives he exact re-quenciesn the testsignal.Frequency s.d. Freq. s.d. Freq. s.d.

True 133.506 267.012 400.518Phase pprox. 133.505 0.654 267.009 1.277 400.514 1.036Quad. fit 132.478 0.4155 265.138 0.771 398.214 0.151

deviation rom the correct requencywas less han 0.01%with this method.We thus conclude hat, for a signalcon-sistingof a singlecomponent, he phasemethod s moreaccurate han the quadratic it.We then determined he effectof "spillover" rom ad-jacentbinsby generating soundconsisting f the sum ofcomponentsn exact bin positions3.1, 6.2, and 9.3. Theresults for 10 frames are found in Table II.

For this case he phasemethod again givesa moreaccurate alue,but the standard eviation s greater.So heconfidencen a singlemeasurement ould be lower for thephasemethod. t shouldbe noted that the actual error isgreater han the standarddeviation or the quadratic it.V. COMPARISON TO THE PHASE VOCODER

A comparison f this methodwith that of the phasevocoder is useful. It should be noted that there are twomethodsof conductingphasevocoderanalyses.With thefilterbankapproach he sinusoidal nalysis unctionsmain-tain a fixedphasewith respect o the signal,and one mea-sures deviations rom center frequencies.On the otherhand, the phasevocoderanalysis s often carried out bytakingFFT's, where he analysisunctions o not maintaintheir initial phase with respect o the input signal, butrather restart at zero for each calculation. In this case theabsolutephase s measured.To compensate,he phasechangecorrespondingo the bin center frequency s sub-tracted from the measuredphasechange,or an equivalentmethod,suchas circularly otatingsamplesn the analysiswindow, s applied.There are, nevertheless,rrors n binsfar removed from the bins in which componentsof thesignalactually fall unless he hop size is 1. Theseerrorsincreaseas the hop size increases o there is a tradeoffbetweendata proliferationand accuracy.Our method s equivalent o a phasevocoderanalysisusing the FFT method with a hop size of 1 sample.Theadvantages that we use he approximation f Eq. (3) anddo not have to perform the secondFFT. Thus we measurethe absolute requency or each bin, and get this informa-tion from a singleFFT. With our methodwe obtain theexact requency f the source or five or so bins on eithersideof the bin with center requencyclosest o that of the

inputsinusoidalomponent. ur method asclearadvan-tages ver he conventionalhase ocoder nd thusholdspromiseor musical ynthesiss well as analysis.VI. CONCLUSION

Our methodof tracking he fundamentalrequency fmusicalpassagesn real time is extremelyaccurateandreports requencieso the nearestquarter tone. Our highresolution requencydetermination an be usedas a backend for a fundamentalrequencyrackerwherehigh pre-cision is desired.Applications ange from analysisofsoundswith continuousrequency ariation o determina-tion of temperamentor performance tudiesn cognitivepsychology.ACKNOWLEDGMENTS

JCB is grateful o IRCAM for their hospitalityduringa Sabbatical leave when much of this work was carried outand to WellesleyCollege or its generous abbaticaleavepolicy.Shewould ike to thankDan Ellis of the MIT Me-dia Lab for invaluable nd illuminating onversations.i-nallyshe s grateful o Jim Beauchampf the University fIllinois for many helpful e-mail discussions.Beauchamp, . W. (1966). "TransientAnalysisof HarmonicMusicalTonesby Digital Computer,"Audio Eng. Soc.PreprintNc,.479.Beauchamp,. W. (1969). "A ComputerSystem or Time-VariantHar-monicAnalysis nd Synthesis f MusicalTones,"Music by Computers,editedby vonFoerster ndBeauchampWiley, New York) :.pp. 19-61.Brown,J. C. (1991). "Calculation f a constantQ spectral ransform," .Acoust. Soc. Am. 89, 425-434.Brown, J. C. (1992). "Musical fundamental requency racking usingapattern ecognition ethod," . Acoust.Soc.Am. 92, 1394-1402.Brown,J. C., and Puckette,M. S. (1992). "An efficient lgorithm or thecalculation f a constantQ transform," . Acoust.Soc.Am. 92, 2698-

2701.Charpentier, . J. (1986). "PitchDetection sing he Short-Term haseSpectrum," roceedingsf the nternationalConferencen Acoustics,Speech,ndSignalProcessingIEEE, New York), pp. 113--116.Dolson,M. (1986). "The PhaseVocoder:A Tutorial," Cornput.MusicJ.10, 14-27.Flanagan, . L., and Golden,R. M. (1966). "PhaseVocoder,"Bell Syst.Tech. J. 45, 1493-1509.Friedman,D. (1985). "Instantaneous-Frequencyistributionvs Time:An Interpretationf the Phase tructure f Speech," roceedingsf theInternationalConferencen Acoustics,peech, nd SignalProcessing,(IEEE, New York), pp. 1121-1124.Grey, J. M., and Moorer,J. A. (1977). "Perceptualvaluationsf syn-

thesizedmusical nstrument tones," J. Acoust. Soc. Am. 62, 454-462.Moorer, J. A. (1978). "The Use of the Phase Vocoder in ComputerMusic Applications," . Audio Eng. Soc.26, 42-45.Oppenheim, . V., andLuis,J. S. (1981). "The Importance f PhasenSignal,"Proc. EEE 69, 529-541.Oppenheim, . V., andSchafer, . W, (1975). DigitalSignalProcessing(Prentice-Hall, EnglewoodCliffs, NJ).Smith,J. O., and Serra,X. (1987). "An Analysis/Synthesisrogram orNon-Harmonic SoundsBasedon a SinusoidalRepresentation," ro-ceedings f the 1987 InternationalComputerMusic ConfirenceInt.ComputerMusicAssoc.,SanFrancisco), p. 290-297.

667 J. Acoust. Soc. Am., Vol. 94, No. 2, Pt. 1, August 1993 J. Brown and M. Puckette: High resolution requency determination 667

a high resolution fundamental frequency determination based on phase changes of the ...

Documents