TRICAP_06

Mohamed HanafiMohamed Hanafi

PLS PATH MODELLING : Computation of latent PLS PATH MODELLING : Computation of latent variables with the estimation mode B variables with the estimation mode B

UNITE DE SENSOMETRIE ET CHIMIOMETRIENantes-France

TRICAP_06

References References

Jan-Bernd Lohmöller, 1989. Latent variable path modelling with partial least squares.Physica-Verlag, Heildelberg

Herman Wold (1985). Partial Least Squares. Encyclopedia of statistical sciences , vol 6 Kotz, S & Johnson, N.L(Eds), John Wiley & Sons, New York, pp 581-591.

TRICAP_06

Data sets Data sets

1X2X mX

Several groups of variables Multiple data sets Multiblock data sets Partitioned matrices

n

p1p2 pm

TRICAP_06

Path Model Path Model

1Xn

p1

4X

p4

n

2X

p2

n

3X

p3

n

Path : • is specified by the investigator• likes to explore a specific point of view from the data• directed graph

TRICAP_06

PLS PM = One principle and two models PLS PM = One principle and two models

All information between blocks of observable is assumed to be All information between blocks of observable is assumed to be conveyed by conveyed by latent variables (linear combination of variables)latent variables (linear combination of variables)..

PrinciplePrinciple

Outer Model ( Factor model, measurement model) relating Manifest variables to their LVshows the manifest variables as depending on the LV

Inner Model(Structural model, Path model) relating endogeneous LV to other LVs shows the LV as dependent on each other

1z

3z 4z

2z

z

TRICAP_06

Real Application : Real Application : European Customer Satisfaction Model (ECSM)European Customer Satisfaction Model (ECSM)

1z

2z

3z 5z

Perceived quality

Customer Expectation

Perceived Value

Custumer satisfaction

4z

Image

6z

Loyalty

7z

Complaints

ECSM is based on well-established theories and applicable for a number of different industries

Fornell, C. (1992).Journal of Marketing, 56, 6-21.

TRICAP_06

PLS PM for two blocksPLS PM for two blocks

1X2Xn n

Applications Applications

Ecology Food science Biospectroscopy Ect….

p1 p2

TRICAP_06

PLS PM for two blocks : models PLS PM for two blocks : models

2z1z

,,...,2,1 ,0 kkjk

kj

kkj pj ezx

uzz 1102 bb

Inner model

Outer Model ( Factor model, measurement model) relating Manifest variables to their LVshows the manifest variables as depending on the LV

z

Inner Model(Structural model, Path model) relating endogeneous LV to other LVs shows the LV as dependent on each other

1z 2z

TRICAP_06

PLS PM for two blocks : Estimation PLS PM for two blocks : Estimation

Estimated parametersEstimated parameters ComputationComputation

Latent variablesLatent variables

IterativeIterative

Outer model Outer model

Inner modelInner model

OLSOLS

1'' kkkk wXXw

kkk wXz

kj

k ,0

2,1k

10 ,bb

Inner and outer models are not estimated simultaneously!!!

TRICAP_06

Computation of latentes variables Computation of latentes variables Two estimation modesTwo estimation modes

1,2 normalize 4

3

A mode

A mode 2

1,2 , 1

1

11

1'2

12

2'1

11

k

k

sk

skk

sk

ss

ss

skk

sk

z

wXz

zXw

zXw

wXz

s1z

ss1

'2

12 zXw

ss1

'2

12

'2

12 zXXXw

MODE A for X2

MODE B for X2

s1z

TRICAP_06

Compact description of the algorithm Compact description of the algorithm

XX11MODE AMODE A MODE BMODE B

XX22

MODE AMODE A

MODE BMODE B

ss

ss

1'2

12

2'1

11

zXw

zXw

ss

ss

1'2

1

2'2

12

2'1

1

1'1

11

zXXXw

zXXXw

ss

ss

1'2

1

2'2

12

2'1

11

zXXXw

zXw

ss

ss

1'2

12

2'1

1

1'1

11

zXw

zXXXw

TRICAP_06

Link with Power Method Link with Power Method

XX11MODE AMODE A MODE BMODE B

XX22

MODE AMODE A

MODE BMODE B

s

ss

2

212

Az

Azz

'11

'22 XXXX

'11

'22 PPPP

'11

'22 PPXX

'11

'22 XXPP

2

1'

kkkk XXXP

TRICAP_06

Link with psychometric methodsLink with psychometric methods

XX11MODE AMODE A MODE BMODE B

XX22

MODE AMODE A

MODE BMODE B

212 ,covmax zz

2

212

var

,covmax

z

zz 212 ,max zzr

1

212

var

,covmax

z

zz

Hotelling H. (1936). Biometrika, 28, 321-377.

Tucker, L. R. (1958). Psychometrika, 23, 111-136.

Van den Wollenberg. A. L. (1977). Psychometrika, 42, 2, 207-219

Canonical correlation

Interbattery method

Redundancy Analysis

Redundancy AnalysisTucker, L. R. (1958). Van den Wollenberg. A. L. (1977).

Hotelling H. (1936).

TRICAP_06

Several blocksSeveral blocks

kjk

kjj

kj ezx 0

kpj 1

mk 1

Outer model

1X2X mXn

p1 p2 pm

TRICAP_06

Inner ModelInner Model

1z

2z

3z3z

4

431,40,44 uzz bb

322,311,30,33 uzzz bbb

TRICAP_06

PLS PM : EstimationPLS PM : Estimation

Estimated parametersEstimated parameters ComputationComputation

Latent variablesLatent variables

IterativeIterative

Outer modeOuter mode

Inner modelInner model

OLSOLS

1'' kkkk wXXw

kkk wXt mk ,....,2,1

parametersparameters

TRICAP_06

Notations Notations

1z

3z 4z

2z

0100

1011

0100

0100

klcC

4321 zzzz

4

3

2

1

z

z

z

z

TRICAP_06

Lohmöller’s procedure (mode B)Lohmöller’s procedure (mode B)

1

11

1'1

1

normalize 5

4

1 3

1 , 2

1 , 1

sk

skk

sk

skkkk

sk

sl

skl

k

lkl

sk

sk

mk

mkc

mk

z

wXz

ZXXXw

zZ

z

Jan-Bernd Lohmöller, 1989. Latent variable path modelling with partial least squares.Physica-Verlag, Heildelberg Chapter 2. page 29.

Factorial SchemeFactorial Scheme Centroid SchemeCentroid Scheme

0,1

0,1sl

sk

sl

sk

kl r

r

zz

zzv lkkl r zz ,

Mode AMode A Mode BMode B

skk

sk

sk

sk ZX

ZZw '

'1 1

skkk

sk 1

'1'1 ZXXXw

TRICAP_06

1z

3z 4z

2z

3311 , zzzZ r 3322 , zzzZ r 2231133 ,, zzzzzzZ rr 4344 , zzzZ r

TRICAP_06

RemarksRemarks

Lohmöller’s procedureLohmöller’s procedure

implemented in various softwares :• PLS Graph (W. Chin)• SPAD • SmartPLS (Ringle and al.)

TRICAP_06

Wold’s procedure (Mode B)Wold’s procedure (Mode B)

1

11

1'1

11

normalize 5

4

3

,)),(()),(( 2

, 1

000

00000

0

0

00

0

00

0

sk

skk

sk

skkkk

sk

sl

sk

sl

kllk

sl

sk

sl

kllk

sk

sk

rsigncrsignc

z

wXz

ZXXXw

zzzzzzZ

z

(1) Herman Wold (1985). Partial Least Squares. Encyclopedia of statistical sciences , vol 6 Kotz, S & Johnson, N.L(Eds), John Wiley & Sons, New York, pp 581-591.

TRICAP_06

RemarksRemarks

Wold’s procedureWold’s procedure

proposed by Wold for • six blocks• Centroid scheme

Extended by Hanafi (2006) • arbitrary number of blocks• take into account the Factorial scheme

Hanafi, M (2006).Computational Statistics.Hanafi, M (2006).Computational Statistics.

TRICAP_06

Computational OverviewComputational Overview

algorithmalgorithm ConvergenceConvergence

Latent variablesLatent variables Iterative Iterative YESYES

Outer modelsOuter models

Inner modelsInner models OLSOLS YESYES

Two blocks

algorithmalgorithm ConvergenceConvergence

Latent variablesLatent variables Iterative Iterative ??

Outer modelsOuter models

Inner modelsInner models

OLSOLS YESYES

No problem

More than two Blocks

No problem

TRICAP_06

Monotony convergence of Wold’s procedure Monotony convergence of Wold’s procedure

m

lklkklm rcf

1,21 ,,...,, zzzzz

112

1121 ,...,,,...,, s

msss

mss ff zzzzzz

m

lklkklm rch

1,

221 ,,...,, zzzzz

112

1121 ,...,,,...,, s

msss

mss hh zzzzzz

Hanafi, M (2006).Computational StatisticsHanafi, M (2006).Computational Statistics

TRICAP_06

Proof : CentroidProof : Centroid

TRICAP_06

Proof : FactorialProof : Factorial

TRICAP_06

TRICAP_06

Not the case for Lohmöller’s procedureNot the case for Lohmöller’s procedure

TRICAP_06

Path for the exemplePath for the exemple

1z

3z

2z

TRICAP_06

CentroidCentroid FactorialFactorial

Wold’s procedureWold’s procedure 79 79 iterationsiterations

73 iterations73 iterations

Lohmöller’ s Lohmöller’ s procedureprocedure

159 159 iterationsiterations

128 iterations128 iterations

TRICAP_06

Lohmöller’s procedure revisitedLohmöller’s procedure revisited

Hanafi and al (2005)

• Update ckk=0 by ckk=1 monotonically convergence of the procedure (Mode B+ centroid scheme)

Hanafi and al (2006) • Alternative procedure

Hanafi, M and Qannari, EM (2005).Computational Statistics and Data Analysis, 48, 63-67Hanafi, M and Qannari, EM (2005).Computational Statistics and Data Analysis, 48, 63-67

Hanafi, M and Kiers, H.A.L. (2006).Computational Statistics and Data Analysis. Hanafi, M and Kiers, H.A.L. (2006).Computational Statistics and Data Analysis.

TRICAP_06

Wold’s procedure depends on starting vectorsWold’s procedure depends on starting vectors

1z

3z

2z

3z

1z

TRICAP_06

Value of the Criterion =7.10

Value of the Criterion =10.28

TRICAP_06

Characterization of latent variablesCharacterization of latent variables

TRICAP_06

Generalized Canonical Correlation Analyses Generalized Canonical Correlation Analyses (CGA)(CGA)

m

kllklkrSUMCOR

,1,

,Max zz

m

kllklkrS

,1,

2 ,Max SQCOR zz

Kettering, J.R. (1971), Bimetrika

[Horst (1965)]

[Kettering (1971)]

An overview for five generalizations of canonical correlation analysis

TRICAP_06

Path model for GCAPath model for GCA

1z

3z

2z

3z

1z

TRICAP_06

PLS PM and Generalized canonical correlation PLS PM and Generalized canonical correlation

SSQCORFactorialBMODE

SUMCORCentroidBMODE

TRICAP_06

ConclusionsConclusions

Two blocksTwo blocks

PLS PM = general framewok for psychometric methodsThe procedures of the computation of the latent variables are equivalent to a power method

More than two blocks ( with mode B for all blocks) More than two blocks ( with mode B for all blocks)

Monotony property of Wold’s procedure Characterization of the latent variable as a solution (among other) of non linear systems of equations Strong link with generalized canonical correlation analysis PLS PM with the estimation mode B can be seen as an extension of CGA.

TRICAP_06

Perspectives Perspectives

To what extend the solutions obtained by wold’s To what extend the solutions obtained by wold’s procedure are at least a local maximum? procedure are at least a local maximum?

Similar results for mode A and mixed mode ? Similar results for mode A and mixed mode ?

Optimisation principle for Latent variables ?Optimisation principle for Latent variables ?

TRICAP_06

Computational OverviewComputational Overview

algorithmalgorithm ConvergenceConvergence OptimalitOptimality y

Latent variablesLatent variables Iterative Iterative YESYES YESYES

Outer modelsOuter models

Inner modelsInner models OLSOLS YESYES

YesYes

No problem

Two blocks

algorithmalgorithm ConvergenceConvergence OptimalitOptimality y

Latent variablesLatent variables Iterative Iterative ?? ??

Outer modelsOuter models OLSOLS YESYES YesYes

Inner modelsInner models OLSOLS YESYES YesYes

No problem

More than two Blocks

TRICAP_06

Characterization of latent variablesCharacterization of latent variables

1' kkuuKk ,,2,1

KKKKK

K

K

u

u

u

u

u

u

AA

AA

AA

22

11

2

1

21

221

112

0

0

0

lkklklc PPu '

2

1'

,

kkk

pnkk

XXXP

Kk 1