Lectures Remote Sensing

PRINCIPAL COMPONENT ANALYSIS

dr.ir. Jan Clevers

Centre of Geo-InformationEnvironmental Sciences

Wageningen UR

Principal Component Analysis (PCA)

• Observation vector• Statistics• PCA• Eigenvectors• Eigenvalues

(L&K6 pp. 527-533)(L&K5 pp. 536-542)

Wageningen UR 2010

x1

x2

xN

Observation vector: X= per pixel k = 1, ...., N features (spectral bands) per pixel

Let us assume that we have n pixels: X1 = , X2 = ,……, Xn =

General: for pixel j, the observation vector is:

Xj= , j = 1, ….., n

x11

x21

xN1

x12

x22

xN2

x1n

x2n

xNn

x1

x2

xN j

Wageningen UR 2010

Spectral Pattern

Wageningen UR 2010

Image Statistics

Mean per spectral band:

n

µk = ∑ xkjj=1

Variance per spectral band:

n

σk2 = ∑ ( xkj - µk )2

j=1

n

(n-1)

µk

σk xk

Wageningen UR 2010

Covariance between 2 spectral bands (k and l):

n

ckl = ∑ ( xkj - µk ) ( xlj - µl ) j=1

Correlation coefficient between 2 spectral bands (k and l):

ckl

ρkl = ----------σk⋅σl

(n-1)

Image Statistics -2-

Wageningen UR 2010

Mean vector: M =

µ1

µ2

µN

c11 c12 ……….. c1N

c21 c22 ………... c2N

cN1 cN2………... cNN

Covariance matrix: C =

The above can be applied to a complete image, but also to individual classes.

Matrix notation

4 -0.340.160.12-0.01

-0.010.310.260.15

-0.390.51-0.38-0.07

1

5 0.410.270.540.54

0.570.410.620.62

0.590.610.560.66

0.140.880.430.27

1

7 0.440.400.750.68

0.430.520.790.69

0.760.710.850.79

-0.610.810.04-0.08

0.610.960.780.84

1

6 -0.200.07.

-0.15

-0.18-0.10.

-0.17

0.24-0.21.

-0.15

-0.72-0.49.0.02

0.16-0.39.0.03

0.64-0.34.0.07

1

1 2 3 4 5 7 6

11

2 0.830.900.920.89

1abcd

3 0.810.840.880.91

0.830.920.900.91

1

Four test sites

(from literature)

Wageningen UR 2010

Correlations Landsat-TM Spectral Bands

Wageningen UR 2010

Linear orthogonal transformation:

(1) translation: X - µ(2) rotation (angle α)

N original correlated variables:x1, x2, ....., xN

PCA

N new uncorrelated variables:p1, p2, ....., pN

x2

x1

α

p1p2x2 - µ2

x1 - µ1

Principal Component Analysis (PCA)

Wageningen UR 2010

N images with one and the same pixel geometry

x1 x2 xNN original images

p1 p2 pN

S synthetic bands

X

P

S < N

Image Space

Wageningen UR 2010

Purpose PCA:

(1) Data reduction: the "information content" of PC-1 (p1) is largest,next PC-2, etc.

(2) Visual interpretation: a colour composite of PC-1, PC-2 and PC-3provides mostly all information present in an image (> 95% of allvariation)

(3) Improvement of image interpretation • separability of classes• spatial differences• monitoring

Which function ?

P = B (X - µ)translation

rotation

Wageningen UR 2010

P = B (X - µ) p1 b11 b12 ..... b1N x1 - µ1

p2 b21 x2 - µ2

= ⋅

pN bN1 bN2 ..... bNN xN - µN

P B X - µ

PCA Calculation -1-

E.g.: 2-dimensional case: p1 = b11 (x1-µ1) + b12 (x2-µ2)p2 = b21 (x1-µ1) + b22 (x2-µ2)

These new p-axes are determined in such a way that they form a decreasing series concerning the part of the total image variation(variance) which they explain.

Wageningen UR 2010

The PCA mostly is calculated based on a representative sample fromthe image space, and subsequently applied to all pixels.

Requirement 1: B is an orthogonal transformation matrix

b1T b11 b12 ..... b1N first eigenvector

b2 b21

B = =

bN bN1 bN2 ..... bNN Nth eigenvectorloadings

PCA Calculation -2-

2 requirements for PCA:

(1) linear, orthogonal transformation

(2) variables p1, p2, ....., pN uncorrelated� covariance-matrix must be diagonal

PCA Calculation -3-

Wageningen UR 2010

Requirement 2: matrix C after PCA is a diagonal matrix

c11 c12 ..... c1N λ1

c21 c22 ..... c2N PCA λ2

cN1 cN2 ..... cNN λN

trace C = trace L

c11 + c22 + ... cNN λ1 + λ2 + ... λN

0

0

PCA Calculation -4-

Wageningen UR 2010

| C - λ ⋅ I | = 0

c11-λ c12 ..… c1N

c21 c22-λ= 0

cN1 cN2 ..… cNN-λ

This PCA is based on the calculation of eigenvectors b1, b2, ... bN and eigenvalues λ1, λ2, ... λN of the covariance matrix C.

The N eigenvalues are calculated byputting the N solutions of the followingdeterminant equal to zero:

For each solution λi, there is aneigenvector bi for which holds:

Subsequently, the vectors bi are normalized, such that their length becomes equal to 1.

( C - λi I ) bi = 0

Individual spectral bands Landsat-TM, Wageningen 1995

Wageningen UR 2010

Principal components of Landsat-TM, Wageningen 1995

Wageningen UR 2010

Summary PCA Calculation

Example 2-dimensional case

(1) Calculate mean vector µ1

µ2

c11 c12

(2) Calculate covariance matrix C = c21 c22

(3) Calculate eigenvalues λ1 and λ2

| C - λ ⋅ I | = 0

c11- λ c12

c21 c22 - λ

(c11- λ) • (c22 - λ) - c12 • c21 = 0

λ1 , λ2

= 0

Wageningen UR 2010

Summary PCA Calculation -2-

(4) Calculate eigenvectors b1 and b2

( C - λ ⋅ I ) ⋅ bi = 0

c11- λ1 c12 b11 0c21 c22 - λ1 b12 0

c11- λ2 c12 b21 0c21 c22 - λ2 b22 0

Then normalize b1 and b2

b1 and b2 are orthogonal and define the direction of p1 and p2 .

= b1

= b2

Wageningen UR 2010

Wageningen UR 2010

Confidence region95%

39%

d√λ1d√λ2

p1

p2

x1

x2

√λ 1

√λ2

√a22=σ22

√a22=σ22

√a11=σ11 √a11=σ11

39% and 95% Confidence Region

µ2

µ1