in it seminar_r_d_mos_cut

54
1 Multidimensional Model Order Selection Multidimensional Model Order Selection

Upload: jpdacosta

Post on 09-May-2015

11.929 views

Category:

Spiritual


0 download

TRANSCRIPT

Page 1: In it seminar_r_d_mos_cut

1

Multidimensional Model Order SelectionMultidimensional Model Order Selection

Page 2: In it seminar_r_d_mos_cut

2

MotivationMotivation

� Stock Markets: One example of [1]

[1] : M. Loteran, “Generating market risk scenarios using principal components analysis: methodological and practical considerations”, in the Federal Reserve Board, March, 1997.

⇒ Information : Long Term Government Bond interest rates. Canada, USA, 6 European countries and Japan.

⇒Result : by visual inspection of the Eigenvalues (EVD). Three main components: Europe, Asia and North America.

Page 3: In it seminar_r_d_mos_cut

3

MotivationMotivation

� Ultraviolet-visible (UV-vis) Spectrometry [2]

[2] : K. S. Von Age, R. Bro, and P. Geladi, “Multi-way analysis with applications in the chemical sciences,”Wiley, Aug. 2004.

Non-identified substanceRadiation

Wav

elen

gth

Oxidation statepH

samples

⇒Result : successful application of tensor calculus .In [2], the model order is estimated via the core consistencyanalysis (CORCONDIA) by visual inspection .

Page 4: In it seminar_r_d_mos_cut

4

MotivationMotivation

� Sound source localization

[3] : A. Quinlan and F. Asano, “Detection of overlapping speech in meeting recordings using the modified exponential fitting test,” in Proc. 15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland.

Microphone array

Sound source 1

Sound source 2

⇒Applications: interfaces between humans and robots and data processing.

⇒MOS: Corrected Frequency Exponential Fitting Test [3]

Page 5: In it seminar_r_d_mos_cut

5

MotivationMotivation

� Wind tunnel evaluation

[4] : C. El Kassis, “High-resolution parameter estimation schemes for non-uniform antenna arrays,” PhD Thesis, SUPELEC, Universite Paris-Sud XI, 2009. (Wind tunnel photo provided by Renault)

⇒MOS: No technique is applied. [4]

Wind

Array

Source: Carine El Kassis [4].

Page 6: In it seminar_r_d_mos_cut

6

Receive array: 1-D or 2-D

FrequencyTime

Transmit array: 1-D or 2-D

Direction of Arrival (DOA)

DelayDoppler shift

Direction of Departure (DOD)

MotivationMotivation

� Channel model

Page 7: In it seminar_r_d_mos_cut

7

MotivationMotivation

� An unlimited list of applications⇒Radar;⇒Sonar;⇒Communications;⇒Medical imaging;⇒Chemistry;⇒Food industry;⇒Pharmacy;⇒Psychometrics;⇒Reflection seismology;⇒EEG;⇒…

Page 8: In it seminar_r_d_mos_cut

8

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 9: In it seminar_r_d_mos_cut

9

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 10: In it seminar_r_d_mos_cut

10

IntroductionIntroduction

� The model order selection (MOS)⇒ is required for the principal component analysis (PCA).⇒ is the amount of principal components of the data.⇒ has several schemes based on the Eigenvalue Decomposition (EVD).⇒ can be estimated via other properties of the data , e.g., removing

components until reaching the noise level or shift invariance property of the data.

� The multidimensional model order selection (R-D MOS) ⇒ requires a multidimensional structure of the data , which is taken into

account (this additional information is ignored by one dimensional MOS ).⇒ gives an improved performance compared to the MOS.⇒ based on tensor calculus , e.g., instead of EVD and SVD, the Higher Order

Singular Value Decomposition (HOSVD) [5] is computed.

[5] : L. de Lathauwer, B. de Moor, and J. Vanderwalle, “A multilinear singular value decomposition”, SIAM J. Matrix Anal. Appl., vol. 21(4), 2000.

Page 11: In it seminar_r_d_mos_cut

11

IntroductionIntroduction� A large number of model order selection (MOS) schemes have been proposed in

the literature. However,⇒ most of the proposed MOS schemes are compared only to Akaike’s

Information Criterion (AIC) [6] and Minimum Description Length (MDL) [6];⇒ the Probability of correct Detection (PoD) of these schemes is a function of

the array size (number of snapshots and number of sensors). � In [7], we have proposed expressions for the 1-D AIC and 1-D MDL. Moreover, for

matrix based data in the presence of white Gaussian noise , the Modified Exponential Fitting Test (M-EFT)⇒ outperforms 12 state-of-the-art matrix based model order select ion

techniques for different array sizes. � For colored noise , the M-EFT is not suitable, as well as several other MOS

schemes, and the RADOI [8] reaches the best PoD according to our comparisons.[6] : M. Wax and T. Kailath “Detection of signals by information theoretic criteria”, in IEEE Trans. on

Acoustics, Speech, and Signal Processing, vol. ASSP-33, pp. 387-392, 1974.[7] : J. P. C. L. da Costa, A. Thakre, F. Roemer, and M. Haardt, “Comparison of model order selection

techniques for high-resolution parameter estimation algorithms,” in Proc. 54th International Scientific Colloquium (IWK), (Ilmenau, Germany), Sept. 2009.

[8] : E. Radoi and A. Quinquis, “A new method for estimating the number of harmonic components in noise with application in high resolution radar,” EURASIP Journal on Applied Signal Processing, 2004.

Page 12: In it seminar_r_d_mos_cut

12

IntroductionIntroduction

� One of the most well-known multidimensional model order selection schemes in the literature is the Core Consistency Analysis (CORCONDIA) [9]⇒ a subjective MOS scheme , i.e., depends on the visual interpretation.

� In [10], we have proposed the Threshold-CORCONDIA (T-CORCONDIA)⇒ which is non-subjective , and its PoD is close, but still inferior to the 1-D AIC and

1-D MDL.� By taking into account the multidimensional structure of the data , we extend the

M-EFT to the R-D EFT [10] for applications with white Gaussian noise .� For applications with colored noise , we proposed the Closed-Form PARAFAC

based Model Order Selection (CFP-MOS) scheme,⇒ which outperforms the state-of-the-art colored noise scheme RADOI [11].

[9] : R. Bro and H.A.L. Kiers. A new efficient method for determining the number of components in PARAFAC models. Journal of Chemometrics, 17:274–286,2003.

[10] : J. P. C. L. da Costa, M. Haardt, and F. Roemer, “Robust methods based on the HOSVD for estimatingthe model order in PARAFAC models,” in Proc. 5-th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2008), (Darmstadt, Germany), pp. 510 - 514, July 2008.

[11] : J. P. C. L. da Costa, F. Roemer, and M. Haardt, “Multidimensional model order via closed-form PARAFAC for arbitrary noise correlations,” submitted to ITG Workshop on Smart Antennas 2010.

Page 13: In it seminar_r_d_mos_cut

13

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 14: In it seminar_r_d_mos_cut

14

Tensor Tensor algebraalgebra

� 3-D tensor = 3-way array

� n-mode products between and

� Unfoldings

M1

M2

M3

“1-mode vectors”

“2-mode vectors”

“3-mode vectors”

i.e., all the n-mode vectors multiplied from the left-hand-side by

11 22

Page 15: In it seminar_r_d_mos_cut

15

The HigherThe Higher --Order SVD (HOSVD)Order SVD (HOSVD)� Singular Value Decomposition � Higher-Order SVD (Tucker3)

“Full HOSVD”

Low-rank approximation (truncated HOSVD)

“Economy size HOSVD”

“Full SVD”

� Matrix data model

signalsignal partpart noisenoise partpart

rank rank dd

� Tensor data model

signalsignal partpart noisenoise partpart

rank rank dd

“Economy size SVD”

Low-rank approximation

Page 16: In it seminar_r_d_mos_cut

16

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 17: In it seminar_r_d_mos_cut

17

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)� Observation is a superposition of noise and signal

⇒ The noise eigenvalues still exhibit the exponential profile [12,13]

⇒ We can predict the profileof the noise eigenvaluesto find the “breaking point”

⇒ Let P denote the number of candidate noise eigenvalues.

• choose the largest Psuch that the P noise eigenvalues can be fitted with a decaying exponential

d = 3, M = 8, SNR = 20 dB, N = 10

[12] : J. Grouffaud, P. Larzabal, and H. Clergeot, “Some properties of ordered eigenvalues of a wishartmatrix: application in detection test and model order selection,” in Proceedings of the IEEEInternational Conference on Acoustics, Speech and Signal Processing (ICASSP’96).

[13] : A. Quinlan, J.-P. Barbot, P. Larzabal, and M. Haardt, “Model order selection for short data: Anexponential fitting test (EFT),” EURASIP Journal on Applied Signal Processing, 2007

Page 18: In it seminar_r_d_mos_cut

18

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)

� Start with P = 1

⇒Predict λM-1 based on λM

⇒Compare this predictionwith actual eigenvalue

⇒ relative distance:

⇒ In our case it agrees, we continue

d = 3, M = 8, SNR = 20 dB, N = 10

Page 19: In it seminar_r_d_mos_cut

19

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)

� Now, P = 2

⇒Predict λM-2 based on λM-1 and λM

⇒ relative distance

d = 3, M = 8, SNR = 20 dB, N = 10

Page 20: In it seminar_r_d_mos_cut

20

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)

� Now, P = 3

⇒Predict λM-3 based on λM-2, λM-1, and λM

⇒ relative distance

d = 3, M = 8, SNR = 20 dB, N = 10

Page 21: In it seminar_r_d_mos_cut

21

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)

� Now, P = 4

⇒Predict λM-4 based on λM-3, λM-2, λM-1, and λM

⇒ relative distance

d = 3, M = 8, SNR = 20 dB, N = 10

Page 22: In it seminar_r_d_mos_cut

22

Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)

� Now, P = 5

⇒Predict λM-5 based on λM-4 , λM-3, λM-2, λM-1, and λM

⇒ relative distance

⇒The relative distance becomes very big, we havefound the break point.

d = 3, M = 8, SNR = 20 dB, N = 10

Page 23: In it seminar_r_d_mos_cut

23

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 24: In it seminar_r_d_mos_cut

24

RR--D Exponential Fitting TestD Exponential Fitting Test

⇒ In the R-D case, we have a measurement tensor

⇒This allows to define the r-mode sample covariance matrices

⇒The eigenvalues of are denoted by for

⇒They are related to the higher-order singular values of the HOSVD of through

� r-mode eigenvalues

Page 25: In it seminar_r_d_mos_cut

25

RR--D Exponential Fitting TestD Exponential Fitting Test

⇒The R-mode eigenvalues exhibit an exponential profile for every R

⇒Assume . Then we can define global eigenvalues

⇒The global eigenvalues also follow an exponential profile, since

⇒The product across modes enhances the signal-to-noise ratio and improves the fit to an exponential profile

� R-D exponential profile

Page 26: In it seminar_r_d_mos_cut

26

RR--D Exponential Fitting TestD Exponential Fitting Test

⇒Comparison between the global eigenvalues profile and the profile of the last unfolding

� R-D exponential profile

Page 27: In it seminar_r_d_mos_cut

27

RR--D Exponential Fitting TestD Exponential Fitting Test

⇒ Is an extended version of the M-EFT operating on the

⇒Exploits the fact that the global eigenvalues still exhibit an exponential profile

⇒The enhanced SNR and the improved fit lead to significant improvements in the performance

⇒ Is able to adapt to arrays of arbitrary size and dimension through theadaptive definition of global eigenvalues

� R-D EFT

Page 28: In it seminar_r_d_mos_cut

28

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 29: In it seminar_r_d_mos_cut

29

� Another way to look at the SVD

⇒ decomposition into a sum of rank one matrices⇒ also referred to as principal components (PCA)

� Tensor case:

SVD and PARAFACSVD and PARAFAC

+ +=

+ +=

Page 30: In it seminar_r_d_mos_cut

30

HOSVD and PARAFACHOSVD and PARAFAC

� HOSVD � PARAFAC

� Identity tensor� Core tensor

• Core tensor usually is full. R-D STE [14] • Identity tensor is always diagonal. CFP-PE [15]

[14]: M. Haardt, F. Roemer, and G. Del Galdo, ``Higher-order SVD based subspace estimation to improve the parameter estimation accuracy in multi-dimensional harmonic retrieval problems,'' IEEE Trans. Signal Processing, vol. 56, pp. 3198 - 3213, July 2008.

[15]: J. P. C. L. da Costa, F. Roemer, and M. Haardt, “Robust R-D parameter estimation via closed-form PARAFAC,” submitted to ITG Workshop on Smart Antennas 2010.

Page 31: In it seminar_r_d_mos_cut

31

ClosedClosed --form solution to PARAFACform solution to PARAFAC

� The task of PARAFAC analysis: Given (noisy) measurements

and the model order d, findsuch that

Here is the higher-order Frobenius norm (sum of squared magnitude of all elements).

[16] :F. Roemer and M. Haardt, “A closed-form solution for multilinear PARAFAC decompositions,” in Proc. 5-th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2008), (Darmstadt, Germany), pp. 487 - 491, July 2008.

� Our approach: based on simultaneous matrix diagonalizations (“closed-form”) .� By applying the closed-form PARAFAC (CFP) [16]

⇒ R*(R-1) simultaneous matrix diagonalizations (SMD) are possible;⇒ R*(R-1) estimates for each factor are possible;⇒ selection of the best solution by different heuristics (residuals of the SMD) is

done

Page 32: In it seminar_r_d_mos_cut

32

� For P = 2, i.e., P < d

ClosedClosed --form PARAFAC basedform PARAFAC basedModel Order SelectionModel Order Selection

+=

+=� Assuming that d = 3, and solutions with the two smallest residuals of the SMD.� Using the same principle as in [17], the error is minimized when P = d.� Due to the permutation ambiguities, the components of different tensors are

ordered using the amplitude based approach proposed in [18].

[17] :J.-M. Papy, L. De Lathauwer, and S. Van Huffel, “A shift invariance-based order-selection technique for exponential data modelling,” in IEEE Signal Processing Letters, vol. 14, No. 7, pp. 473 - 476, July 2007.

[18] :M. Weis, F. Roemer, M. Haardt, D. Jannek, and P. Husar, “Multi-dimensional Space-Time-Frequency component analysis of event-related EEG data using closed-form PARAFAC,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP), (Taipei, Taiwan), pp. 349-352, Apr. 2009.

� For P = 4, i.e., P > d

+ += +

+ += +

Page 33: In it seminar_r_d_mos_cut

33

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 34: In it seminar_r_d_mos_cut

34

ComparisonsComparisons

Page 35: In it seminar_r_d_mos_cut

35

ComparisonsComparisons

Page 36: In it seminar_r_d_mos_cut

36

ComparisonsComparisons

Page 37: In it seminar_r_d_mos_cut

37

ComparisonsComparisons

Page 38: In it seminar_r_d_mos_cut

38

ComparisonsComparisons

Page 39: In it seminar_r_d_mos_cut

39

ComparisonsComparisons

Page 40: In it seminar_r_d_mos_cut

40

ComparisonsComparisons

Page 41: In it seminar_r_d_mos_cut

41

ComparisonsComparisons

Page 42: In it seminar_r_d_mos_cut

42

ComparisonsComparisons

Page 43: In it seminar_r_d_mos_cut

43

ComparisonsComparisons

Page 44: In it seminar_r_d_mos_cut

44

ComparisonsComparisons

Page 45: In it seminar_r_d_mos_cut

45

ComparisonsComparisons

Page 46: In it seminar_r_d_mos_cut

46

ComparisonsComparisons

Page 47: In it seminar_r_d_mos_cut

47

ComparisonsComparisons

Page 48: In it seminar_r_d_mos_cut

48

ComparisonsComparisons

Page 49: In it seminar_r_d_mos_cut

49

ComparisonsComparisons

Page 50: In it seminar_r_d_mos_cut

50

ComparisonsComparisons

Page 51: In it seminar_r_d_mos_cut

51

ComparisonsComparisons

Page 52: In it seminar_r_d_mos_cut

52

OutlineOutline

� Motivation� Introduction� Tensor calculus� One dimensional Model Order Selection

⇒ Exponential Fitting Test� Multidimensional Model Order Selection (R-D MOS)

⇒Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)

� Comparisons� Conclusions

Page 53: In it seminar_r_d_mos_cut

53

ConclusionsConclusions� State-of-the-art one dimensional and multidimensional model order selection

techniques were presented;

� For one dimensional scenarios:

⇒ in the presence of white Gaussian noise• Modified Exponential Fitting Test (M-EFT)

⇒ in the presence of severe colored Gaussian noise

• RADOI� For multidimensional scenarios:

⇒ in the presence of white Gaussian noise

• R-dimensional Exponential Fitting Test (R-D EFT)⇒ in the presence of colored noise

• Closed-form PARAFAC based Model Order Selection (CF P-MOS) scheme

� The mentioned schemes are applicable to problems with a PARAFAC data model, which are found in several scientific fields.

Page 54: In it seminar_r_d_mos_cut

54

Thank you for your attention!Thank you for your attention!Vielen Dank fVielen Dank f üür Ihre Aufmerksamkeit!r Ihre Aufmerksamkeit!