Journal of Classification 8:93-98 (1991)

Some Notes on the Diagonalization of the ExtendedThree-Mode Core Matrix

Piet Brouwer Pietcr M. Kroonenbcrg

Leiden University Leiden University

Abstract: We extend previous results of Krooncnberg and de Leeuw (1980) andKroonenbcrg (1983, Ch. 5) on transformations of the extended core matrix of theTucker2 model (Kroonenbcrg and de Leeuw 1980). In particular, it is shown thatnon-singular transformations to diagonalize the core matrix will lead to PARAFACsolutions (Harshman 1970; Harshman and Lundy 1984), if such solutions exist.

Keywords: Three-mode principal component analysis; Parallel factor analysis;PARAFAC; CANDECOMP.

1. Introduction

Many models to analyze three-way data contain a set of parameters todescribe relationships between the components (factors, dimensions) of thevarious ways. These parameters are collected in a (possibly) diagonal three-way 'core matrix' (Tucker 1966) when all three ways are reduced, or an'extended core matrix' (Tucker 1972) when only two ways are reduced. (Inthis note we will restrict ourselves to extended core matrices, and for simpli-city we will often refer to them as 'core matrices'.) In some three-waymodels such core matrices arc restricted to be diagonal by definition. Core

This note is a revised version of a paper presented at the 4th European Meeting of thePsychometric Society, Cambridge, U.K., 2-5 July 1985, and is based on the first author's mas-ter thesis.

Requests for reprints and the TUCKALS computer programs should be addressed toPieter M. Kroonenbcrg, Vakgroep Algemene Pedagogiek, Department of Education, LeidenUniversity, P.O. Box 9555, 2300 RB Leiden, The Netherlands.

94 P. Brouwer and P. M. Krooncnbcrg

matrices of non-diagonal models contain far more parameters than diagonalones. As shown by Carroll and Wish (1974; see also Krooncnbcrg 1983, Ch.3; Kicrs 1988) models with diagonal core matrices arc nested within modelswith more general core matrices. One might therefore wonder whether it ispossible to transform more general core matrices into (approximately) diago-nal ones, and thus achieve greater parsimony without much loss of fit of themodel to the data.

The idea of diagonalizing core matrices of three-way models originatesin the multidimensional scaling literature (sec e.g. de Leeuw and Pruzansky1978, and their references). Based upon an idea of Jan de Leeuw, Krooncn-bcrg (1983, Ch. 5) suggested a diagonalization procedure of the core matrixusing non-singular transformations, which is essentially equivalent to apply-ing the PARAFAC/CANDECOMP algorithm (Harshman, 1970; Carroll &Chang, 1970) to the core matrix. In this note we will explore this diagonaliza-tion in greater detail.

2. Theory

Diagonalization of Extended Core Matrices

The Tuckcr2 model (a slight generalization of a model for individualdifferences scaling proposed by Tucker 1972; sec Kroonenbcrg and deLeeuw, 1980) is a way to analyze a three-way data matrixZ = (Z\,Z2,... ,ZK) of the order / xJ x K, where Zk is the £-th frontal / xjslice with / not necessarily equal to J. The Tuckcr2 model is given as

Zk=AGkB' (1)

in which A and B arc I x P and J x Q component matrices of the first andsecond way respectively, and Gk is the k-\h P x Q slice of the extended corematrix, which is of the order P x Q x K. In the TUCKALS2 algorithm usedto solve the estimation of the Tucker2 model, the A and B arc initially ortho-normal matrices, while there arc no restrictions on the Gk. However, theorthonormality restriction functions only to achieve an attractive estimationprocedure, and can be relaxed as soon as a solution has been found. Forinstance to facilitate interpretation, the components may be transformed non-singularly with the inverse transformations applied to the core matrix. Analternative is to simplify the core matrix itself by diagonalizing it. Anextended core matrix is called diagonal if each Gk is a square diagonal matrixof order P x P. (Note that there is no requirement for the Zk to be symmetricor square.) Apart from the advantage of parsimony, Harshman (1970; Harsh-man and Lundy 1984) has shown that in the present context there is only one

Notes on Diagonalization 95

solution for which the core matrix is diagonal (apart from trivial reflectionsand permutations). Moreover, the associated components are unique as well.Therefore, there is no transformational freedom with diagonal core matrices.Harshman's parallel factor analysis model (PARAFAC), in fact, starts withthe restriction that the extended core matrix is diagonal, i.e. the PARAFACmodel can be written as

Zk = ADkB' (k = },...,K), (2)

where A and B are defined as before (without the orthonormality restriction)and the Dk arc diagonal P x P matrices. One way to diagonalize the corematrix would be to use a procedure which approximates a PARAFAC solu-tion analogously to the procedure of e.g. Cohen (1974) to transform a solutionof Carroll and Chang's (1970) non-diagonal individual differences scalingmodel IDIOSCAL to a diagonal INDSCAL solution. The parallel case herewould be to seek matrices Dk, S and 7 such that the function

\\Gk-SDkT'\\ (3)

is minimi/.cd for all k ( k = 1 , . . . , K ) simultaneously, which is a decomposi-tion of the core matrix. Simplification of the core matrix by nonsingulartransformations as in (3) is the same as using the extended core matrix as datafor PARAFAC extracting as many factors as there are components for/4 andB, as we will show below.

Suppose we have data which perfectly fit the PARAFAC model

Zk = ADkB ' (4)

with/1 and B not necessarily orthogonal, and Dk diagonal.Let ULV' be the Singular Value Decomposition (SVD) of A and P<t>Q

the SVD of B. Substitution into (4) gives

Zk = UAV'DkQ4>P' (5)

By defining A* = U,B* = P, and G\ = AV DkQ<b (5) can be written as

Zk=AtG*kB*' (6)

with A* and ß* columnwise orthonormal and G\ not necessarily diagonal.However, this is the Tuckcr2 description of the same data. We may concludethat if the data fit the PARAFAC model perfectly, they will also fit theTuckcr2 model perfectly. What is more, nonsingular matrices 5* = AV and

96 P. Brouwer and P. M. Kroonenbcrg

exist, and diagonal matrices D\ = D* such that

In other words, the perfect Tucker2 solution can be transformed by aPARAFAC decomposition of the extended core matrix into an overallPARAFAC solution. The uniqueness of the nonsingular transformation of theextended core matrix follows from the uniqueness of the overall PARAFACsolution itself. (Sec Harshman 1972).

Both PARAFAC and Tucker solutions, ZPARAFNC and Z-Tucten may beseen as subspaces of the space spanned by the data Z. In case of perfectPARAFAC data both subspaces are identical to Z. If the data^ perfectly fitTucker2 but not PARAFAC with the same dimensionality then Zp^^c is areal subspace of both Z and Zfucker. In general, if ZPAKAfAc £ ̂ Tucker wc maywrite

Z Tucker = ZpARAFAC + £/MRAMC. ZpAK/y.*Ac n E PARAFAC = 0

From this, from the proof given before, and from the uniqueness theorem, itfollows that applying PARAFAC to the extended core matrix of TUCKALS2will produce the same solution as PARAFAC on the data itself for theZ PARAFAC part and no solution for the E PARAFAC part. In other words, giventhat an overall PARAFAC solution exists, and that the correspondingPARAFAC space is a subspace of the Tucker space, the same overall solutionwill be found when PARAFAC is applied to the extended core matrix, evenwith real, noisy data.

If the PARAFAC fit comes close to the TUCKALS2 fit then off-diagonal elements of the core matrix will be (near) zero, and the interpreta-tion can be a fully-Hedged PARAFAC one. If the off-diagonal elements do notvanish, and ipso facto the PARAFAC fit is significantly lower than theTUCKALS2 fit, a PARAFAC interpretation may be less appropriate.

3. Numerical Results

In this section we will illustrate the theoretical results above using theTongue Shape data published in Harshman, Ladcfoged, and Goldstein (1977).

The data consist of the displacement of 13 points of the tongue (modeB), measured during the enunciation of 10 English vowels (mode A) by 5speakers (mode C). We used four versions of these data:

1 . TS - 2f, the original data;

Notes on Diagonalization 97

TABLE 1

Tongue Shape Data: Comparison of the Fit of TUCKALS2 plus CoreDiagonalization with that of PARAFAC

Overall

Data set

1.1.3./. .

TS--2fTS--2pTS--2eTS--3p

01d1

T2+CD

.9260

.0000

.8539

.0000

PAR.

0.92621.00000.85391.0000

0000

Fit

Factor 1

T2+CD

.4735

.4628

.4494

.4623

PAR.

0.47530.46270.45020.4623

Factor

T2+CD

00n0

.3691 0

.3696 0

.3984 0

.3376 0

2

PAR.

.3684

.3696

.3980

.3377

Factor 3

T2+CD PAR.

0.0956 0.0956

Note: The fit per factor is expressed as its root mean squared contributionto the solution.

2. TS - 2p, perfect data, reconstructed from a two-factor PARAFACsolution of 1;

3. TS - 2e, data obtained by adding uniformly distributed error to 2;4. TS - 3p, perfect data, reconstructed from a three-factor PARAFAC

solution of 1.

Table 1 shows in paired columns the overall fit and the fit per factor forthe TUCKALS analysis + core diagonalixation (T2 + CD; left-hand column)and PARAFAC (PAR.; right-hand column).

The three solutions of each PARAFAC analysis were identical, and thesquared correlations between the factor loadings of PARAFAC and TUCK-ALS2 + Core Diagonalization (72 + CD) varied between 0.9996 and 1.0000.Similar results have been obtained for other data sets. Moreover, there aredata sets were the extended core matrix cannot be diagonalizcd, but thesquared correlations between 72 + CD and PARAFAC components are .911or higher. Furthermore, for the data sets investigated, it turned out that when-ever PARAFAC could not find a convergent solution neither could 72 + CD,and similarly if a solution was found the same one was found by both pro-cedures. Given that it was our aim to diagonalize core matrices rather thandevelop an alternative program for PARAFAC, we did not conduct extensivespeed comparisons between TUCKALS2 with core diagonal i zation andPARAFAC. For the analyses run, the impression was that 72 + CD was atleast as fast as PARAFAC.

98 P. Brouwer and P. M. Krooncnbcrg

4. Discussion

Both theoretical and empirical results show that it can be fruitful toinvestigate the transformation to diagonality of the extended core matrix ofthe Tuckcr2 model after a TUCKALS2 solution has been obtained. Theincreased parsimony of a diagonal core matrix and the unique orientation ofcomponents associated with the diagonal core matrix support this contention.Moreover, in some of the data sets examined, it was possible to find a diago-nal core matrix within an essentially nondiagonal one. This allows the possi-bility of assessing how and where the core matrix is not diagonal. There is noguarantee, as is the case for PARAFAC itself, that a solution can be found.

ReferencesCARROLL, J. D., and CHANG, J. J. (1970). "Analysis of Individual Differences in Mult idi-

mensional Scaling via an N-way Gcnerali/.ation of "Eckart - Young" Decomposition,"Psychometrika, 35, 283-319.

CARROLL, J. D., and Wish, M. (1974), "Models and Methods for Three-way Multidimen-sional Scaling," in Contemporary Developments in Mathematical Psychology (Vol. II),Eds. D. H. Krant/., R. C. Atkinson, R. D. Luce, P. Suppes, San Francisco: Freeman, 57-105.

COHEN, L. (1974), "Three-mode Rotation to Approximate INDSCAL Structure (TRIAS),"Paper presented at the Psychometric Society Meeting, Palo Alto, CA.

DE LEEUW, J., and Pruzansky, S. (1978), "A New Computational Method to Fit theWeighted Euclidean Distance Model," Psychometrika, 43, 479-490.

HARSHMAN, R. A. (1970), "Foundations of the PARAFAC Procedure: Models and Condi-tions for an "Explanatory" Multi-mode Factor Analysis," UCLA Working Papers inPhonelis, 16, 1-84 (Xerox University Microfilms, Ann Arbor, MI; 10,085).

HARSHMAN, R. A. (1972), "Determination and Proof of Minimum Uniqueness Conditionsfor PARAFAC1," UCLA Working Papers in Phonetics, 22, 111-117 (Xerox UniversityMicrofilms, Ann Arbor, MI; 10,085).

HARSHMAN, R. A., LADEFOGED, P., and GOLDSTEIN, L. (1977), "Factor Analysis ofTongue Shapes," Journal of the Acoustical Society of America, 62, 693-707.

HARSHMAN, R. A., and LUNDY, M. E. (1984), "The PARAFAC Model for Three-way Fac-tor Analysis and Multidimensional Scaling," in Research Methods for Mullimode DataAnalysis, Eds. H. G. Law, C. W. Snydcr Jr., J. A. Hattic, and R. P. McDonald, New York:Pracger, 122-215.

KIERS, H. A. L. (1988), "Comparison of "Anglo-Saxon" and "French" Three-modeMethods, Statistique et Analyse des Données, 13, 14-32.

KROONENBERG, P. M. (1983), Three-mode Principal Component Analysis: Theory andApplications, Leiden, The Netherlands: DSWO Press.

KROONENBERG, P. M., and DE LEEUW, J. (1980), "Principal Component Analysis ofThree-mode Data by Means of Alternating Least Squares Algorithms," Psychomelrika,45, 69-97.

TUCKER, L. R (1966), "Some Mathematical Notes on Three-mode Factor Analysis,"Psychometrika, 31, 279-311.

TUCKER, L.R (1972), "Relations Between Multidimensional Scaling and Three-mode FactorAnalysis," Psychometrika, 37, 3-27.