SHORT-TIME MULTICHANNEL NOISE CORRELATION MATRIX ESTIMATORS FOR

ACOUSTIC SIGNALS

By: Jonathan Blanchetteand

Martin Bouchard

Overview▶ Introduction▶ Framework▶ Noise correlation matrix estimators▶ Performance measure▶ Conclusion & Outlook

Introduction

▶ Speech enhancement or beamforming algorithms require the noise Power Spectral Density (PSD).

▶ Many multichannel noise PSD estimation algorithms require some knowledge on sound sources:

• Number of sources (Many sources are possibly present , time varying)

• Directivities (Can be unknown)

▶ Additionally many assume that the noise field is diffuse and homogeneous (The noise field could be inhomogeneous).

Motivation

Framework

▶ General equation:

Noise correlation matrix models

Framework

▶ General equation:

Noise correlation matrix models

Geometry dependent part

Framework

▶ General equation:

▶ Geometry dependent part is named spatial spectral matrix:

Noise correlation matrix models

Geometry dependent part

Framework

▶ General equation:

▶ Geometry dependent part is named spatial spectral matrix:

Noise correlation matrix models

Geometry dependent partscalar

Framework Cont’d

▶ Spatial spectral matrix based model:

Noise correlation matrix models

Geometry dependent part

scalar

Framework Cont’d

▶ Coherence based equation:

Noise correlation matrix models

Framework Cont’d

▶ Coherence based equation:

Noise correlation matrix models

Geometry dependent part

Framework Cont’d

▶ Coherence based equation:

▶ Geometry dependent part is the coherence matrix with elements:

Noise correlation matrix models

Geometry dependent part

Framework Cont’d

▶ Coherence based equation:

▶ Geometry dependent part is the coherence matrix with elements:

Noise correlation matrix models

Geometry dependent part

Framework Cont’d

▶ Coherence based equation:

▶ Geometry dependent part is the coherence matrix with elements:

Noise correlation matrix models

Geometry dependent partNoise PSD

Framework Cont’d

▶ In free field, for spherical or cylindrical isotropic noise field:

Models equivalence in special cases

Noise correlation matrix estimators

▶ Assuming sources to be statistically independent to the noise:

Noisy signal correlation matrix estimate

Noisy signal correlation matrix

Noise correlation matrix estimators Cont’d

▶ Assuming sources to be statistically independent to the noise:

Noisy signal correlation matrix

Sources correlation matrix

Noise correlation matrix estimators Cont’d

▶ Assuming sources to be statistically independent to the noise:

or

▶ Generalized EigenValue Problem (GEVP)!

Noisy signal correlation matrix

Noise correlation matrix estimators Cont’d

▶ We spatially decorrelate the noise (assuming =):

GEVP

Noise correlation matrix estimators Cont’d

▶ We spatially decorrelate the noise (assuming =):

▶ Eigenvalue decomposition:

GEVP

Noisy signal correlation matrix decomposition

Noise correlation matrix estimators Cont’d

▶ Signal subspace eigenvectors:▶ Signal subspace eigenvalues:

Signal subspace

Noise correlation matrix estimators Cont’d

▶ Noise subspace eigenvectors:▶ Noise subspace eigenvalues:

Noise subspace

Noise correlation matrix estimators Cont’d

Γ̂𝑤

Noise correlation matrix estimators Cont’d

Γ̂𝑤

Noise correlation matrix estimators Cont’d

Γ̂𝑤

Noise correlation matrix estimators Cont’d

�̂�𝑤=𝚽1/2 [𝑽 𝑥+𝑤𝑽 𝑤 ] [ Γ̂𝑤 𝑰 �̂�×𝑁 0

0 �̂�w][¿𝑽 𝑥+𝑤

𝐻

¿𝑽 𝑤𝐻 ]𝚽1/2

Γ̂𝑤=tr ( Λ̂w )𝑀 −𝑁

Noise correlation matrix estimation

Noise correlation matrix estimators Cont’d

▶ Equivalent formula:

Noise correlation matrix estimation

Noise correlation matrix estimators Cont’d

▶ Equivalent formula:

Noise correlation matrix estimation

Project onto the noise subspace

Noise correlation matrix estimators Cont’d

▶ Equivalent formula:

Noise correlation matrix estimation

Project onto the signal subspace

Noise correlation matrix estimators Cont’d

▶ can be interpreted as a time-frequency dependent multi-channel Voice (or Source) Activity Detector (VAD)

▶ is estimated with short data records model order estimator: o The Akaike Information Criterion (AIC)

[Aikaike,1974]

o Modified Minimum Description Length (MDL)[R. C. Hendriks, J. Jensen and R. Heusdens,2008](with heuristically chosen to work for stationary noise in anechoic room)

Noise correlation matrix estimation

Noise correlation matrix estimators Cont’d

▶ The case set to 1 includesNoise correlation matrix estimation

[Kamkar-Parsi and Bouchard, 2009]

Performance measure

▶ Other multichannel algorithms that can’t be included as a subcases involve knowledge on the sources directivities. Not fair!

▶ Single channel algorithms don’t use information on directivities.

Comparison with single channel algorithms

Performance measure

▶ Problems with comparison:o Single channel algorithms estimate only

diagonal elements of the correlation matrix

Comparison with single channel algorithms

Performance measure Cont’d

▶ Problems with comparison:o Reference noise-only correlation matrix

includes diffuse noise + some occasional interferences

o Algorithm treats these interferences as sources (not included in )

o Effective number is not known

Comparison with reference noise correlation matrix

Performance measure Cont’dChannels

Image courtesy of [Kayser et al., 2009]

Performance measure Cont’dChannels

2

4

1

3 56

78

Performance measure Cont’d

▶ TIMIT database used for the sentences▶ Oldenburg university database use for diffuse

noise and HRTFs

Setup

[Kayser et al., 2009]

1 23

Performance measure Cont’dAnechoic environment Log-error with constant SNRs

0 dB 5 dBSensor set φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE{1,2}, N=1 1.1 1.1 1.1 0.6 2.7 3.1 1.8 1.3 1.3 1.8 0.9 2.6 3.0 1.9{3,7}, N=1 1.1 1.0 1.0 0.7 2.7 3.1 1.8 1.3 1.2 1.6 0.8 2.6 3.0 1.9

{3,5,7}, N=2 3.4 1.2 1.1 0.7 2.6 3.1 1.9 3.3 1.3 1.8 0.8 2.6 3.0 2.1{3,4,7,8}, N=3 1.3 1.3 1.4 0.8 2.6 3.1 2.1 1.5 1.5 2.4 1.0 2.7 3.1 2.4

10 dB 15 dBSensor set φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE{1,2}, N=1 1.4 1.4 2.7 1.1 2.5 3.1 2.1 1.6 1.6 3.7 1.3 2.5 3.6 2.3{3,7}, N=1 1.4 1.3 2.4 1.0 2.5 3.1 2.1 1.6 1.5 3.4 1.2 2.5 3.6 2.3

{3,5,7}, N=2 3.2 1.4 2.9 1.0 2.7 3.2 2.5 3.1 1.6 4.2 1.2 3.0 3.9 3.0{3,4,7,8}, N=3 1.8 1.7 3.8 1.2 2.8 3.3 2.8 2.0 1.9 5.6 1.4 3.3 4.1 3.5

Performance measure Cont’dAnechoic environment Log-error time varying SNRs

Sensor set φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE{1,2}, N=1 1.5 1.4 3.1 1.9 2.9 3.7 3.2{3,7}, N=1 1.4 1.3 2.9 1.6 2.9 3.7 3.2

{3,5,7}, N=2 3.2 1.5 3.3 1.8 3.2 3.9 3.6{3,4,7,8}, N=3 1.8 1.7 4.4 2.6 3.4 4.0 4.0

Performance measure Cont’dCafeteria environment Log-error with N=1 for binaural setting

SNR φ,AIC ψ,AIC φ,MMDL ψ,MMDL MS Dobl. MMSE0 dB 1.7 1.7 1.8 2.2 3.9 3.8 2.45 dB 2.0 2.0 2.6 2.3 3.7 3.8 2.6

10 dB 2.4 2.4 3.7 2.6 3.6 4.3 3.015 dB 3.0 3.0 5.0 3.1 3.7 5.4 3.5

Conclusion & Outlook

▶ Subspace projection noise correlation matrix estimator presented:o Independent on knowledge of sourceso Performance depends on estimate o Framework presented for any diffuse noise

field type and geometryo Performance measures are competitive

▶ Further work:o Develop adaptive coherence estimatorso Test more short data records model order

estimation schemes