merlot 2007 – sra: an online research tool for spectral and roughness analysis of sound signals -...

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University

SRA: An online research tool for spectral and roughness analysis of

sound signals

Pantelis N. Vassilakis - DePaul UniversityChicago, USA

MERLOT 2007New Orleans


1. Auditory roughnessConcept & Models

2. Spectral AnalysisNew versus old methods

3. SRAa. Outlineb. Examples of use

At A Glance


Auditory Roughness

Auditory Roughness: Harsh, raspy sound quality of narrow harmonic intervals.

An acoustic/sensory dimension of dissonance. One of the perceptual manifestations of interference, expressed as a function of a complex signal’s spectral distribution; a dimension of timbre

_ Adding two sine signals with frequencies f1 and f2 results in a complex signal whose amplitude fluctuates between a minimum and a maximum value at a rate equal to |f1-f2|.

_ Amplitude fluctuation rate:a) < ~15 fluctuations/second → Beating b) between ~15 and ~75-150 fluctuations/second → Roughnessc) > ~ 150 fluctuations/second → combination tones, envelope pitch, etc..


• Helmholtz (1885)

• Plomp & Levelt (1965)

• Kameoka & Kyriyagawa (1969a,b)

• Hutchinson & Knopoff (1978)

• Daniel & Weber (1997)

• Sethares (1998)

• Leman (2000) • Pressnitzer, McAdams &

Colleagues (1997, 1999a, 1999b, 2000) (auditory periphery mechanisms models)

Previous roughness calculation models


)()( 122121 ffsbffsb eeR

b1 = 3.5 b2 = 5.75

x* = 0.24

s1 = 0.0207 s2 = 18.96

211

*

sfs

xs

Roughness, frequency separation, and register


• General assumption:

Roughness is proportional to A1 * A2

Roughness and amplitude

• Previous experimental studies:

von Béckésy (1960)

Terhardt (1974)


Amplitude modulation depth versus degree of amplitude fluctuation


Proposed Roughness Calculation Model

Roughness estimation model (sine pairs): R = X * Y * Z (Vassilakis, 2001, 2005)

X = (Amin*Amax)0.1 Dependence of R on the absolute amplitude of the sines (Terhardt, 1974; Vassilakis, 2001)

Y = 0.5 [2Amin / (Amin+Amax )]3.11

Dependence of R on the relative amplitudes of the sines (von Béckésy, 1960; Terhard, 1974; Vassilakis, 2001)

Z = e-b1s(fmax - fmin) – e-b2s(fmax - fmin) [b1 = 3.5; b2 = 5.75; s = 0.24/(s1fmin + s2); s1 = 0.0207; s2 = 18.96]Dependence of R on relative (frequency difference of the sines) and absolute (frequency of the lower sine) frequencies of the added sines (Kameoka &

Kuriyagawa, 1969a&b; Plomp & Levelt, 1965; Sethares, 1998)


Comparison of 3 roughness calculation models


Drawbacks of traditional FFT

Spectral Analysis

Frequency and time values returned are forced to fit onto the time-frequency grid defined by the analysis window

Frequency/temporal “smearing” and uncertainty on precise energy values


Frequency analysis and spectral peak pickingSTFT algorithm based on the Reassigned Bandwidth-Enhanced Additive Model

Spectral Analysis

Dual STFT, fine-tuning spectral analysis results _ Frequency: time derivative of the argument (phase) of the complex analytic signal representing a given frequency bin _ Time: frequency derivative of the STFT phase, defining the local group delay (correction that pinpoints the precise excitation time)

Theory:

Developed by Kodera et al. (1976)

Expressed mathematically by Auger & Flandrin (1995)

Implemented to sound spectral analysis by Fulop & Fitz (2006a,b; 2007)


SRA online

http://www.acousticslab.org/roughness

http://www.acousticslab.org/roughness/index.html


Dissonance & Orchestration


Roughness / Tesion Profile of the Improvisation on the Mijwiz

0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)

Roughness Profile (7-point moving average; 250ms)

Roughness Profile



0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)


Tension Profile - Improviserr=0.422

Roughness & Tension Profiles



0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)


Tension Profile - Improviser

Tension Profile - Listenersr=0.068

r=0.422

Roughness & Tension Profiles

http://www.aip.org/149th/Vassilakis.html


ReferencesAuger, F. and Flandrin, P. (1995). "Improving the readability of time frequency and time scale representations by the reassignment method," IEEE

Transactions on Signal Processing 43: 1068-1089.

von Békésy, G. (1960). Experiments in Hearing. New York: Acoustical Society of America Press (1989).

Daniel, P. and Weber, R. (1997). "Psychoacoustical roughness: Implementation of an optimized model," Acustica 83: 113-123.

Fitz, K. and Haken, L. (2002). "On the use of time-frequency reassignment in additive sound modeling," Journal of the Audio Engineering Society 50(11): 879-893.

Fitz, K., Haken, L., Lefvert, S., Champion, C., and O'Donnell, M. (2003). "Cell-utes and flutter-tongued cats: Sound morphing using Loris and the Reassigned Bandwidth-Enhanced Model," Computer Music Journal 27(4): 44-65.

Fulop, S. A. and Fitz, K. (2006a). "Algorithms for computing the time corrected instantaneous frequency (reassigned) spectograms with applications," J. Acoust. Soc. Am. 119(1): 360-371.

Fulop, S. A. and Fitz, K. (2006b). "A spectrogram for the twenty-first century," Acoustics Today 2(3): 26-33.

Fulop, S.A. and Fitz, K. (2007). "Separation of components from impulses in reassigned spectrograms," J. Acoust. Soc. Am. 121(3): 1510-1518.

Helmholtz, H. L. F. 1885 [1954]. On the Sensations of Tone as a Physiological Basis for the Theory of Music. 2nd English edition. New York: Dover Publications. [Die Lehre von den Tonempfindungen, 1877. 4th German edition, trans. A. J. Ellis.]

Kameoka, A. and Kuriyagawa, M. (1969a). "Consonance theory, part I: Consonance of dyads," J. Acoust. Soc. Am. 45(6): 1451-1459.

Kameoka, A. and Kuriyagawa, M. (1969b). "Consonance theory, part II: Consonance of complex tones and its calculation method," J. Acoust. Soc. Am. 45(6): 1460-1469.

Kendall, R. A. (2002). Music Experiment Development System (MEDS) 2001B for Windows. Los Angeles: University of California Los Angeles, Department of Ethnomusicology, Program in Systematic Musicology.

Plomp, R. and Levelt, W. J. M. (1965). "Tonal consonance and critical bandwidth," J. Acoust. Soc. Am. 38(4): 548-560.

Pressnitzer, D. and McAdams, S. (1997). "Influence of Phase Effects on Roughness Modeling," ICMC: International Computer Music Conference, Thessaloniki, Greece, September 1997

Pressnitzer, D. and McAdams, S. (1999a). "Two phase effects on roughness perception, ". J. Acoust. Soc. Am. 105(5): 2773-2782.

Pressnitzer, D. and McAdams, S. (1999b). Summation of roughness across frequency regions. In: Dau T, Hohmann V, and Kollmeier B (Eds) Temporal processing in the auditory system: Psychophysics, physiology and models of hearing, pages 105-108. World Scientific Publishing, Singapore.

Pressnitzer, D., McAdams, S., Winsberg, S., and Fineberg, J. (2000). " Perception of musical tension for non-tonal orchestral timbres and its relation to psychoacoustic roughness," Perception and Psychophysics, 62(1): 66-80.

Sethares, W. A. (1998). Tuning, Timbre, Spectrum, Scale. London: Springer-Verlag.

Terhardt, E. (1974). "On the perception of periodic sound fluctuations (roughness)," Acustica 30(4): 201-213.

Vassilakis, P. N. (2001). Perceptual and Physical Properties of Amplitude Fluctuation and their Musical Significance. Doctoral Dissertation. Los Angeles: University of California, Los Angeles; Systematic Musicology.

Vassilakis P. N. (2005). "Auditory roughness as a means of musical expression," Selected Reports in Ethnomusicology 12 (Perspectives in Systematic Musicology): 119-144.

merlot 2007 – sra: an online research tool for spectral and roughness analysis of sound signals -...

Documents

spectral analysis results

absolute frequency

frequency derivative

timefrequency grid

frequency separation

absolute amplitude

dependence of r

time values