matrices ii

1

Matrices II

SOLO HERMELIN

Updated: 20.07.07

2

SOLO Matrices

Table of Content

Singular Values

Definitions

Domain and Codomain of a Matrix A

Properties of Square Orthogonal Matrices

Definition of the Singular Values

Geometric Interpretation of Singular Values

Properties of Singular Values

Moore-Penrose Pseudoinverse Matrix Anxm † of Amxn .

Householder Transformation

Projection of a vector on a vector . b

a

111 min&min1

nxmxnxmxnxxbxAd

nx

Computation of Moore-Penrose Pseudoinverse Matrix, A †

Properties of Moore-Penrose Pseudoinverse Matrix, A †

Description of Projections Related to Moore-Penrose Pseudoinverse

Particular case (1) r = n ≤ m:

Particular case (2) r = m ≤ n:

3

SOLO Matrices

Table of Content (continue – 1)

General Solution of Amxn Xnxp = Bmxp

Particular case (1) r = m ≤ n

Particular case (2) r = n ≤ n

General Solution of YpxmAmxn = Cpxn

Particular case (1) r = m ≤ n

Particular case (2) r = n ≤ n

Inverse of Partitioned Matrices

Matrix Inverse Lemmas Identities

Matrix Schwarz Inequality

Trace of a Square Matrix

References

4

SOLO Matrices

Singular Values

Definitions:

Any complex matrix A with n rows (r1, r2,…,rn) and m columns (c1,c2,…,cm)

m

n

nxm ccc

r

r

r

A ,,, 21

2

1

can be considered as a linear function (or mapping or transformation) for am-dimensional domain to a n-dimensional codomain.

AcodomyAdomxxAyA nxmxnxm 11;:

In the same way its conjugate transpose:

H

n

HH

H

m

H

H

H

mxn rrr

c

c

c

A ,,, 212

1

is a linear function (or mapping or transformation) for an-dimensional codomain to a m-dimensional domain.

AcdomxAcodomyyAxA mxnx

HH

mxn 111111 ;:Table of Contents

5

SOLO Matrices

Domain and Codomain of a Matrix A

The domain of A can be decomposed into orthogonal subspaces:

ANARAdom H

HAR

AN

HAN

AR

xAy

11 yAx H

Adomxmx 1

11mxx

Acodomy nx 11

1nxyR (AH) – is the row space of AH (dimension r)

N (A) – is the null-space of A (x N (A) A x = 0) or the kernel of A (ker (A)) (dimension m-r)

The codomain of A (domain of AH) can be decomposed into orthogonal subspaces:

HANARAcodom

R (A) – is the column space of A (dimension r)

N (AH) – is the null-space of AH (dimension n-r)

Singular Values

Table of Contents

6

SOLO

Hermitian = Symmetric if A has real components

Hermitian Matrix: AH = A, Symmetric Matrix: AT = A

Matrices

Properties of Square Orthogonal Matrices

Use Pease, “Methods of Matrix Algebra”, Mathematics in Science and Engineering Vol.16, Academic Press 1965

Definitions:

Adjoint Operation (H):

AH = (A*)T (* is complex conjugate and T is transpose of the matrix)

Skew-Hermitian = Anti-Symmetric if A has real components.

Skew-Hermitian: AH = -A, Anti-Symmetric Matrix: AT =-A

Unitary Matrix: UH = U-1, Orthonormal Matix: OT = O-1

Unitary = Orthonormal if A has real components.

Charles Hermite1822 - 1901

7

SOLO Matrices

Properties of Square Orthogonal Matrices (continue – 1)

Lemma1: All the eigenvalues of a hermitian matrix H are real and the eigenvectors are orthogonal.

Proof of Lemma1:

Pre-multiply by :iii xHx Hix i

Hiii

Hi xxHxx

and take the conjugate transpose: iH

iiiHH

i

H

iH

i xxxHxHxx *

This proves that the eigenvalues of H are real.

Subtract those two equations: 00 ** i

H

iiii

H

iii xxsincexx

From Hij

Hjj

Hj

HHj

Hj xxHxHxHx *

Pre-multiply by and post-multiply by and subtract iii xHx H

jx Hij

Hj xHx ix

0

i

H

jji

i

H

jji

H

j

i

H

jii

H

jxx

xxHxx

xxHxx

0 iH

jji xxIf

If we can use the Gram-Schmidt procedure to obtain an eigenvector orthogonal to .

ji jx~

ix i

iH

i

jH

ijj x

xx

xxxx ~

8

SOLO Matrices


Lemma1: All the eigenvalues of a hermitian matrix H are real and the eigenvectors are orthogonal.

Proof of Lemma1 (continue – 1):

0

i

H

jji

i

H

jji

H

j

i

H

jii

H

jxx

xxHxx

xxHxx

0 iH

jji xxIf

If we can use the Gram-Schmidt procedure to obtain an eigenvector orthogonal to .

ji jx~

ix i

iH

i

jH

ijj x

xx

xxxx ~

we can see that 0~ i

Hi

iH

i

jH

ij

Hij

Hi xx

xx

xxxxxx

jii

i

H

i

j

H

i

jiii

i

H

i

j

H

i

jji

i

H

i

j

H

i

jj xxxx

xxxx

xx

xxxxH

xx

xxxHxH ~~

q.e.d.

9

SOLO Matrices


Lemma2: Any hermitian matrix H can be factored in H = U Λ UH

where Λ=diag (λ1,λ2,…,λn) and U is unitary i.e. U UH = UH U = In.Proof of Lemma2:

Let normalize the orthogonal eigenvectors of H ;i.e. iii xxu /:

or H U = U Λ where U = [u1,u2,…,un]

Because U is a square matrix having orthonormal columns, and is a square matrix,U is also a unitary matrix satisfying UH U=U UH=In. q.e.d.

000

00

00

,,,,,, 2

1

2121

nn uuuuuuH

10

SOLO Matrices


Lemma3: A AH and AH A are hermitian matrices that have the same nonzero real positive eigenvalues.

Proof of Lemma3:

q.e.d.

HHHHHH AAAAAA i.e. A AH is hermitian, therefore the eigenvalues λi (A AH) are real and positive according to Lemma 1.

Suppose ui is a normalized eigenvector of λi (A AH) ≠0

i

H

ii

H uAAuAA Pre-multiply by AH and define i

H

H

i

i uAAA

v

1:

i

H

ii

H

H

i

i

HH

iH

i

i

HH

i

HH

ii

HH

vAAvAA

AA

uAAA

AA

uAAAuAAAuAAA

we get

We can see that νi is the eigenvector of AH A and λi (A AH) is the corresponding eigenvalue, meaning that both AH A and A AH have the same nonzero eigenvalues.

From 0

2

2

i

i

i

H

i

i

H

iH

ii

H

i

H

ii

HH

ii

H

ii

H

v

vA

vv

vAvAAAvvAAvAAvvAAvAA

Therefore we can define 0: H

ii AA

11

SOLO Matrices


Lemma4: If U is a unitary matrix then all its eigenvalues have unit modulus. .

Proof of Lemma4:

form the inner product

IUUUU HH Consider the set of eigenvalues x1, x2, …, xn which we know to be complete and

iii xUx

nixxIxxUxUxxx iiiiH

iiH

iiHH

iiH

iii ,11**

q.e.d.

12

SOLO Matrices


Proof of Lemma5:

Lemma5: Every unitary matrix U can be expressed as an exponential matrix: where H is hermitian (jH is skew-hermitian)

jHeU

Since the eigenvalues of U have unit modulus; i.e. we can writenii ,11

niej HijHiHii ,1sincos

jHjjj eSeeediagSU HnHH 1,,, 21

121 ,,, SdiagSH HnHH where:

q.e.d.

13

SOLO Matrices


Table of Contents

Decomposition of Square Matrices:

HHHH AAj

jAAAAAAA22

1

2

1

2

1

HH

H AAAA

2

1

2

1

HHH

H AAAAAA

2

1

2

1

2

1

here: Hermitian

Skew-Hermitian

HHH

H AAj

AAj

AAj

222Hermitian

HH AAj

jAAA22

1 the matrix generalization of the decomposition of a complex number in the real and imaginary part.

14

SOLO Matrices

Lemma6: (6.1) Every complex nxm matrix of rank can be factored into:


H

H

rmxrnxrrn

rmrxH

mxmnxmnxnnxm

xmrm

rxmrxr

rnnxnxr V

VUUVUA

2

11

21 00

0

where 0,,, 21211 rrdiagrxr

Unxn and Vmxm are unitary matrices, i.e.:

rnxrn

rxr

H

H

H

H

H

H

I

IUU

U

UUU

U

UUUUU

0

021

2

1

2

121

rmxrm

rxr

H

H

H

H

H

H

I

IVV

V

VVV

V

VVVVV

0

021

2

1

2

121

(6.2) σi i=1,…,r are the positive square roots of the nonzero eigenvalues of AH A or A AH and are called the singular values of A.

(6.3) The dyadic expansion of A is: where ui and vi are the columns of U1 and V1 respectively.

r

i

Hiii vuA

1

Singular Values

15

SOLO Matrices

Lemma6 (continue – 1):


(6.5) The columns of V are orthonormal eigenvectors of AH A:

rmxrmxrrm

rmrxH rxrVVVVAA00

02

1

2121

i.e. the columns of V1 are the eigenvectors of the nonzero eigenvalues, and the columns of V2 are the eigenvectors of the zero eigenvalues of AH A.

(6.6) The following relations exist between U1 and V1

1111

1111

rxrnxrmxr

rxrmxrnxr

UAV

VAUH

nxm

nxm

i.e. the columns of U1 are the eigenvectors of the nonzero eigenvalues, and the columns of U2 are the eigenvectors of the zero eigenvalues of A AH.

(6.4) The columns of U are orthonormal eigenvectors of A AH:

rmxrnxrrn

rmrxH rxrUUUUAA00

021

2121

Singular Values

16

SOLO Matrices



HAR

AN

HAN

AR

xAy

11 yAx H

Adomxmx 1

11mxx

Acodomynx

11

1nxy

(6.7) The columns U1 of form an orthonormal basis for the column space of A: ARUR 1

The columns of U2 form an orthonormal basis for the nullspace of AH: HH AANUR ker2

The columns of V1 form an orthonormal basis for the column space of AH:

HARVR 1

The columns of V2 form an orthonormal basis for the nullspace of A:

AANVR ker2

Singular Values

17

SOLO Matrices


Proof of Lemma 6:


and

From Lemma 3 we have

H

H

rnxrnxrrn

rnrxH

nxnnxn

H

mxnnxm

xnrn

rxnrxr

rnnxnxrnxn U

UUUUUAA

2

12

1

21100

0

H

H

rmxrmxrrm

rmrxH

mxmmxmnxm

H

mxn

xmrm

rxmrxr

rmmxmxrmxm V

VVVVVAA

2

12

1

21200

0

where 0,,, 21211 rrdiagrxr

and riAAAA H

i

H

ii ,,2,10:

Those equations can be rewritten as: 0

0

1121

1121

VVVAA

UUUAAH

H

or

0

0

2

2

111

111

VAA

UAA

VVAA

UUAA

H

H

H

H

HU 2

HV2

00

00

22222

22222

VAVAVAVAAV

UAUAUAUAAUHHH

HHHHHH

Singular Values

18

SOLO Matrices


Proof of Lemma 6 (continue – 1):


02 UAH The columns of U2 form an orthonormal basis for the nullspace of AH:

HH AANUR ker2

02 VA The columns of V2 form an orthonormal basis for the nullspace of A:

AANVR ker2

111 UUAA H The columns U1 of form an orthonormal basis for the column space of A:

ARUR 1

111 VVAAH The columns of V1 form an orthonormal basis for the column space of AH:

HARVR 1

HAR

AN

HAN

AR

xAy

11 yAx H

Adomxmx 1

11mxx

Acodomy nx 11

1nxy

Singular Values

19

SOLO Matrices



Definition of the Singular Values HAR

AN

HAN

AR

xAy

11 yAx H

Adomxmx 1

11mxx

Acodomy nx 11

1nxy

From

riuAuAAA

v i

H

i

i

H

H

i

i ,,2,111

:

we have

r

r

H

r uuuAvvv

/100

0/10

00/1

2

1

2121

or 1

111

UAV H

11

1

1

2

11

1

111

2111

UUUAAVAUUAA

H

H

riuuuAAvA iiii

i

i

H

i

i ,,2,111 2

from which 1

111

VAU rivAu i

i

i ,,2,11

111 VAU H

Singular Values

20

SOLO Matrices



Definition of the Singular Values HAR

AN

HAN

AR

xAy

11 yAx H

Adomxmx 1

11mxx

Acodomy nx 11

1nxy

Using and let compute AH A V 111 VAU H02 VA

rmxrnxrrn

rmrx

nxm

H

nxm

H

nxm

H

nxm

H

nxmH

H

mxmnxm

H

nxn

rxr

rmmxxrrnmxrxrrn

rmmxrxnmxrrxn

rmmxmxr

xnrn

rxn

VAUVAU

VAUVAUVVA

U

UVAU

00

01

2212

2111

21

2

1

From this equation we obtain:

r

i

H

iii

H

H

H

rmxrnxrrn

rmrx

nxm

vuVU

V

VUUA

rxmrxrnxr

xmrm

rxmrxr

rnnxnxr

1

111

2

11

21 00

0

Singular Values

Table of Contents

21

SOLO Matrices

Let perform the following transformations in the domain and codomain :

Geometric Interpretation of Singular Values

Suppose, first, that A is square and r = n = m, and consider the spherical hypersurfacein the domain of A for which:

1v 1x

2x

2v

1x

111 vAv

111 xAx

222 xAx

222 vAv

The Indicator Ellipsoid of a 2 x 2 Matrix11

11

nxnxnnx

mxmxmmx

Uy

Vx

Because y = A x: UVVUVAUy H

From which: 11 mxnxmnx

11

22

2

2

2

r

i

i

HHH VVxxx

From we have and the mapping of the sphericalhypersurface, in the codomain of A is the hypersurface of an ellipsoid:

11 mxnxmnx iii /

11

2

r

i i

i

This ellipsoid is called the indicator ellipsoid of A and the singular values arethe lengths of the principal axes of this ellipsoid.

Singular Values

22

SOLO Matrices

If the square matrix A is singular, i.e., r < n = m, the indicator ellipsoid shrinks to zero in the directions of the principal axes vi for which σi = 0. In this case:

Geometric Interpretation of Singular Values (continue – 1)

01 1

1

2

nr

r

i i

i

If the general case of nonsquare matrices with r < n ≠ m, if we choose the cylindrical hypersurface that has a circular hypersurface projection in R (AH):

01 1

1

2 mr

r

i

i

then its mapping will be the surface of the ellipsoid in R (A).

01 1

1

2

nr

r

i i

i

HAR

AN

HAN

AR

Singular Values

Table of Contents

23

SOLO Matrices


(1) The maximum singular value of Anxm is:

2

2

02121maxmaxmaxmax:

22 x

xAxAxAAA

xxxii

(2) The minimum singular value of Anxm is:

2

2

02121minminminmin:

22 x

xAxAxAAA

xxxii

Proof of (1) and (2)

Using x = V ζ we can write:

m

i

ii

HxV

HHHHH xVVxxAAxxAxAxA1

22222

2

11

22

2

m

i

i

HxV

HHH xVVxxxx

To obtain the maximum/minimum of that satisfy the condition we construct the Hamiltonian by adjoin the constraint to the extremum problem:

2

2xA 12

2x

1:,1

2

1

22m

i

i

m

i

iiH + for maximum- for minimum

Singular Values

24

SOLO Matrices

Properties of Singular ValuesProof of maximum/minimum singular value of Anxm (continue – 1)

The necessary conditions for extremum are:

1:,1

2

1

22m

i

i

m

i

iiH + for maximum- for minimum

miH

ii

i

,,2,102 2

Kuhn-TuckerCondition

miminimumfor

maximumforHi

i

,,2,10

02 2

2

2

Maximization problem solution: AH 22

1,max with

0

10,1

2

1

1

2

21

m

i

im

Kuhn-Tucker Condition

Minimization problem solution: AH m

22,min with

0

10,1

2

1

2

11

m

m

i

imm

Kuhn-Tucker Condition

Singular Values

25

SOLO Matrices

Properties of Singular ValuesProof of maximum/minimum singular value of Anxm (continue – 2)

For any x ≠ 0 we have:

0max

max

2

2

2

2

2

xx

xIAAxAA

xx

xAAxA

x

xAH

HHH

H

HH

The inequality holds because (AHA-I λmax [AHA]) is non-positive definite.

We can see that the equality is satisfied for x = eigenvector (AHA) that correspondsto λmax [AHA], therefore:

0max2

2

0

A

x

xAx

In the same way, or any x ≠ 0 we have:

0min

min

2

2

2

2

2

xx

xIAAxAA

xx

xAAxA

x

xAH

HHH

H

HH

We can see that the equality is satisfied for x = eigenvector (AHA) that correspondsto λmin [AHA], therefore:

0min2

2

0

A

x

xAx

Ax

xAx

2

2

0max

Ax

xAx

2

2

0min

Singular Values

26

SOLO Matrices


(3) is a norm of Anxm, because it satisfies the norm properties: A

(3.1) is non-negative and if and only if A = 0.

Proof of (3.1):

From Lemma 3:

A 0A 00 AA

(3.2) Multiplication by a complex constant α: AA

(3.3) Triangle Inequalities:

(3.4) Schwarz Inequality:

BABABA

BABA

0A

000 HVUAA

Proof of (3.2):

AAAAxx

2121

22

maxmax

Singular Values

27

SOLO Matrices



(3.1) is non-negative and if and only if A = 0. A 0A 00 AA




BABABA

BABA

Proof of (3.3): BAxBxA

xBxAxBABA

xx

xx

2121

22121

22

22

maxmax

maxmax

BBABBAA From which

BABA

In the same way BAAB BABA

Proof of (3.4):

BAx

xB

y

yA

x

xB

xB

xBA

x

xBABA

xy

xx

2

2

02

2

0

2

2

2

2

02

2

0

maxmax

maxmax

Hermann Amandus Schwarz

1843 - 1921

Singular Values

28

SOLO Matrices


(4) The absolute value of the eigenvalues of a square matrix Anxn are bounded between the minimum and the maximum singular values:

niAAA i ,,2,1

Proof of (4):

We have: 02

2 xAx

xAA

If xi is any normalized eigenvector: A xi = λi xi, then

nix

x

x

x

x

xAi

i

ii

i

ii

i

i ,,2,12

2

2

2

2

2

Therefore: niAAA i ,,2,1

Singular Values

29

SOLO Matrices

Properties of Singular Values(5) A square matrix Anxn is singular iff its minimal singular value is zero.

0 ASingularA Proof of (5):

AAVdiagUA n

H

n 2121 ,,,

Therefore: 00det AAASingularA n

n

i

i

H

n VdiagUA11

21

1

det,,,detdetdet

(6) For a nonsingular square matrix Anxn we have

11

1&

1

A

AA

ArNonsingulaA

Proof of (6):

H

n

H

n UdiagVAVdiagUA /1,,/1,/1,,, 21

1

21

1

1

1

21 /1/10 AAAA nn

Hence: 11

1&

1

A

AA

A

Singular Values

30

SOLO Matrices


(7) If the square matrix (A+B) is singular then the maximum singular values of A and of B are greater or equal than the minimum singular value of B and A, respectively. The opposite is not true.

ABBASingularBA &

Proof of (7):

If (A+B) is singular, there exists a normalized eigenvector u (║u║2=1), s.t.:

22

0 uBuAuBuAuBA From this equation we obtain:

BxBuBuAxAAxx

212221

22

minmax

AxAuAuBxBBxx

212221

22

minmax

To prove that the opposite is not true, consider a counterexample:

15&3430

05

10

04

ABBABA

The right side is satisfied, but is nonsingular.

40

01BA

Singular Values

31

SOLO Matrices


ABBASingularBA &

Proof of (8):

(8) A sufficient condition that the square matrix (A+B) is nonsingular is:

We just proved:

ABorBAingularonsNBA

The proof follows directly from property (7). If (A+B) is singular then

ABBA & ; hence if

then (A+B) is nonsingular.

ABorBA


Singular Values

32

SOLO

To prove this we will consider the following three cases:- (A+B) singular, - (A+B) nonsingular but A and B are singular, - (A+B) nonsingular but A or B, or both are nonsingular.

Matrices


(9) The minimum singular value of a square matrix (A+B) satisfies the inequalities:

ABBABAABBA ,min,max

Proof of (9):

(9.1) - (A+B) singular

According to property (5) 0 BASince (A+B) is singular use property (7)

BABAAB 0

BAABBA 0

This completes the proof when (A+B) is singular.

Singular Values

33

SOLO

(9.2) - (A+B) nonsingular but A and B are singular,

Matrices



ABBABAABBA ,min,max

Proof of (9) (continue – 1):

22

0

uBuBAuBuBuAuBA

If A is singular, ,there exists a normalized eigenvector u (║u║2=1), s.t. A u=0: 0A

ABBxBuBuBAxBABAxx

212221

22

maxmin

and

BABABA

In the same way for (A+B) nonsingular and B singular:

ABBAAB

This completes the proof when (A+B) is nonsingular but A and B are singular.

Singular Values

34

SOLO

(9.3) - (A+B) nonsingular but A or B, or both are nonsingular.

Matrices



ABBABAABBA ,min,max


BAC :Suppose that (A+B) and A are nonsingular, and define:

Pre-multiply by C-1 and post-multiply by A-1:1111 ABCCA

Let take any norm of this equation and write triangle an Schwarz inequalities:

BACABC

ABCCAABCC

1111

1111111

BACCABACC 1111111 B

ABAB

A

111

111

Using property (3), we can define , and because property (6) the previousequation is equivalent to:

** BABABA

If B is nonsingular in the same way we can prove that: ABBAAB

This completes the proof when (A+B) is nonsingular but A or B, or both are nonsingular.

Singular Values

35

SOLO

Using this and property (3.3):

Matrices


(10) If the square matrix A is a big matrix relative to the square matrix B, then (A+B) can be approximated by A:

ABAABABABAIf &

Proof of (10):

We have: BBAA

ABAABABABAA

BABABA

Using: and property (9): BBAA

ABBABAABBA ,min,max

we have:

ABAABABABAA

Singular Values

36

SOLO Matrices


(11) Multiplicative Inequalities for square matrices:

BABABABABABA &

Proof of (11):

The proof is given in the following steps:

(11.1) is the Schwarz inequality of property (3) BABA

(11.2) prove that : BABA

If A or B is singular ( or is zero) then A B is singular ( det [A B] =

det [A].det [B]=0 and ) and we have equality.

A B 0BA

If A or B is nonsingular then A B is nonsingular ( det [A B] =

det [A].det [B]≠0 ) and: 11111111 ABABBAABBA

We define , and use the property (6): ** 11

1&

1

A

AA

A

to obtain:

BABABA

BA 111

111

This result is opposite to Schwarz inequality, proving that is not a norm. A

Singular Values

37

SOLO Matrices



BABABABABABA &


(11.3) prove that :

If A is singular then:

BABAorBA

BABA 0

If A is nonsingular then:

BABA

BAA

BAABAAB

1111

If B is singular then: BABA 0

If B is nonsingular then:

BABAB

BABBABBAA

11

Singular Values

38

SOLO Matrices



BABABABABABA &


(11.4) prove that :

If A or B are singular then A B is singular, and:

BABAorBA

BAorBABA 0If B is nonsingular then:

BABAB

BABBABBAA

11

We also have:

BABA

B

BABBABBABBAA

111

If A is nonsingular then:

BABAA

BABAABAAB

11

We also have:

BABA

A

BABAABAABAAB

111

q.e.d.

Singular Values

39

SOLO Matrices


(12) Any unitary matrix U (U UH = UH U = I) has all the singular values equal to 1.

Proof of (12):

iIUUU i

H

ii 12/12/1

(13) If U is a unitary matrix (U UH = UH U = I) then:

Proof of (13):

iAUAAU iii

iAAAAUUAAUAUAU i

H

i

HH

i

H

ii 2/12/12/1

iAAAAAUUUAAUUAUAUA i

H

i

HH

i

HH

i

H

ii 2/12/12/12/1

q.e.d.

Singular Values

q.e.d.

40

SOLO Matrices

Properties of Singular Values - Summary

(4) The absolute value of the eigenvalues of a square matrix Anxn are bounded between the minimum and the maximum singular values:

niAAA i ,,2,1


(3.1) is non-negative and if and only if A = 0. A 0A 00 AA




BABABA

BABA

(1) The maximum singular value of Anxm is:

2

2

02121maxmaxmaxmax:

22 x

xAxAxAAA

xxxii

(2) The minimum singular value of Anxm is:

2

2

02121minminminmin:

22 x

xAxAxAAA

xxxii

Singular Values

41

SOLO Matrices

Properties of Singular Values – Summary (continue – 1)

(5) A square matrix Anxn is singular iff its minimal singular value is zero.

0 ASingularA

(6) For a nonsingular square matrix Anxn we have

11

1&

1

A

AA

ArNonsingulaA


ABBABAABBA ,min,max

(8) A sufficient condition that the square matrix (A+B) is nonsingular is:

ABorBAingularonsNBA

ABBASingularBA &


Singular Values

42

SOLO Matrices

Properties of Singular Values – Summary (continue – 2)

(12) Any unitary matrix U (U UH = UH U = I) has all the singular values equal to 1.

(13) If U is a unitary matrix (U UH = UH U = I) then: iAUAAU iii


BABABABABABA &

(10) If the square matrix A is a big matrix relative to the square matrix B, then (A+B) can be approximated by A:

ABAABABABAIf &

Singular Values

Table of Contents

43

SOLO Matrices

Householder Transformation

n̂

xnn T ˆˆ

xnn T ˆˆ

x

'x

O A

We want to compute the reflection ofover a plane defined by the normal 1ˆˆˆ nnn T

x

From the Figure we can see that:

xHxnnIxnnxx TT ˆˆ2ˆˆ2'

1ˆˆˆˆ2: nnnnIH TT

We can see that H is symmetric:

HnnInnIH TTTT ˆˆ2ˆˆ2

In fact H is also a rotation of around OA so it must be orthogonal, i.e.HTH=H HT=I.

x

InnnnnnInnInnIHHHH TTTTTT ˆˆˆˆ4ˆˆ4ˆˆ2ˆˆ21


Table of Contents

Alston Scott Householder1904 - 1993

44

SOLO Matrices

The same result is obtained if we compute α that minimizes:

a

b

Pa

ab

p

We want to find such that pba

ap

Projection of a vector on a vector . b

a

or: baaaabapba TTTT 1

0

and: bPbaaaabaaaaap TTTT

11

TT aaaaP 1

: Projection Matrix

aababbabababd TTTT

22

2 2minminminmin

baaa

aad

aabad

TT

T

TT

1

min

2

22

2

0

022

Properties of Projection Matrix

(1) P is idempotent P2 = P

(2) P is symmetric PT = P

cbcPIcPcbP

,

Proof: cbcPIPbcPIbP TT

T ,0

0 PIPTHence: PPP TT PPPPP TTT 2PPPPP TT

b

bP

cP

c


Table of Contents

45

SOLO

Note: If A and b were real, instead of H (transpose & complex conjugate) wehave only T (transpose).

Matrices

Given: Amxn of rank (Amxn) = r ≤ min (m,n) and1mxb

Find: such that is minimal1nxx

11 mxnxmxn bxAd

If the solution is not unique choose such that is minimal1nxx

1nxx

Solution:

The minimum is obtained when

0

0

2

22

2

11

2

AAx

d

bAxAAbxAAbxAx

d

H

HHH

mxnxmxn

bAAAx HH 1

A unique solution exists if AHA is positive definite, or rank (AHA) = n, or det|AHA| ≠ 0

1111

2

11

2

mxnxmxn

H

mxnxmxnmxnxmxn bxAbxAbxAd

Analytic:


111 min&min1

nxmxnxmxnxxbxAd

nx

46

SOLO Matrices

Given: Amxn of rank (Amxn) = r ≤ min (m,n) and1mxb

Find: such that is minimal1nxx

11 mxnxmxn bxAd

If the solution is not unique choose such that is minimal1nxx

1nxx

Solution (continue – 1):

Geometric:

We have A x R (A) for all x domain (A).

We want to find x0 domain (A), such that is normal to A x.

0xAbpb

AdomainxxAbpbxA

0

AdomainxxAAbAxxAbxA HHHH

000

Hence: 00 bAxAA HH

HAR

AN

HAN

ARxA

Adomx 0

AcodomY

Null Space of AKer (A)

span by VA2H

Row Space of Aspan by VA1

H

Column Space of Aspan by UA1

Left Null Space of Aspan by UA2

b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA


111 min&min1

nxmxnxmxnxxbxAd

nx

47

SOLO Matrices

Let decompose as0x

HAR

AN HAN

ARxA

Adomx 0

AcodomY


span by VA2H


H



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

NR

N

H

R

NR xxAofspaceNullANx

AofspaceRowARxxxx 00

0

0

000

Therefore:

RNR xAxAxAxAp 0

0

000

Hence if:

(a) N (A) = 0 or(b) The rows of A are linearly dependent or

(c) rank (A) = r < m

(d) AHA is singular there are a infinity of solutions NRNR xxxxx 00000

The norm of is:0x

NR

xx

NR xxxxxNR

00000

00

Hence: 0&min 000 NR xxx


111 min&min1

nxmxnxmxnxxbxAd

nx

48

SOLO Matrices


TAR

AN

HAN

ARxA

Adomx 0

AcodomY


span by VA2T


T



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

bAx R

00&min 000 NR xxx

Define the Linear Transformation (Matrix),that gives from , as the Pseudoinverseof A. (A is the direct transformation that gives from :

Rx0

b

p

x

xAp

bAx †

R

0

A† is called Moore-Penrose Pseudoinverse Matrix, because was defined independently by E.H.Moore in 1920 and Roger Penrose in 1955.

Eliakim HastingsMoore

1862 - 1932

Roger Penrose1931 -

111 min&min1

nxmxnxmxnxxbxAd

nx

Table of Contents

49

SOLO Matrices

Moore-Penrose Pseudoinverse Matrix Anxm † of Amxn . a

HAR

AN HAN

ARxA

Adomx 0

AcodomY


span by VA2H


H



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

bAx †

R

0

bAx †

R

0


Perform Singular Value Decomposition (S.V.D.) of Amxn:

where 0,,, 21211 rrA diagrxr

UAmxm and VAnxn are unitary matrices, i.e.:

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

H

AA

H

AA

H

AAH

A

H

A

AA

rmxrm

rxr

AAH

A

H

A

A

H

A UUUUUUU

UUU

I

IUU

U

UUU

2211

2

1

2121

2

1

0

0

H

AA

H

AA

H

AAH

A

H

A

AA

rnxrn

rxr

AAH

A

H

A

A

H

A VVVVVVV

VVV

I

IVV

V

VVV

2211

2

1

2121

2

1

0

0

50

SOLO Matrices

HAR

AN HAN

ARxA

Adomx 0

AcodomY


span by VA2H


H



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

bAx †

R

0

Since the norm is invariant to the product of orthogonal matrices

bUxVbxVUUbxVUbxA HHHHH

Introduce the new unknown: R

H

N

H

R

HH xVxVxVxVy

:

But 00

N

H

R

H

N

H

R

H

NR xVxVUxVxVUxAxAxA

RR

I

HH

RR

HH xxVVxxVxVy

2/1

bUybUybxA H

rnxrmxrrm

r

rnrx

y

H

yx

00

0

0

0

minminmin

1



51

SOLO Matrices

HAR

AN HAN

ARxA

Adomx 0

AcodomY


span by VA2H


H



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

bAx †

R

0

Therefore:

R

HH xVxVy

:

RR

I

HT

RR

HH xxVVxxVxVy

2/1

bUybUybxA H

rnxrmxrrm

r

rnrx

y

H

yx

00

0

0

0

minminmin

1

any

xrm

rxH

rmxrnxrrn

r

rmrx

XbUy1

1

1

0

00

/10

0

0/1

Rxyx 0minmin

R

HH

rmxrnxrrn

r

rmrx

xVbUy 0

1

00

/10

0

0/1

bAbUVx †H†

R

0

H†† UVA



52

SOLO Matrices

HAR

AN HAN

ARxA

Adomx 0

AcodomY


span by VA2H


H



b

Rx0

Nx0

0xAp

pb

xA

Adomx

0xA

bAx †

R

0

Where:

rmxrnxrrn

r

rmrx

†

nxm

00

0

0

0

: 1

1

1

H

mxm

†

nxmnxn

†

nxm UVA



H

AAAH

A

H

A

rmxrnxrrn

rmrxA

AnA

†

nxm rxmrxrnxr

xmrm

rxmrxr

rnnxmxrUV

U

UVVA 1

1

11

2

11

1

2100

0:

1

1

1

1

0

0

:

r

1-

A rxr

Table of Contents

53

SOLO Matrices



rnxrnxrrn

rnrxrxr

rnxrmxrrm

r

rnrx

rmxrnxrrn

r

rmrx

mxn

†

nxm

I

00

0

00

0

0

0

00

0

0

01

1

1

1

rmxrmxrrm

rmrxrxr

rmxrnxrrn

r

rmrx

rnxrmxrrm

r

rnrx

†

nxmmxn

I

00

0

00

0

0

0

00

0

0

0

1

1

11

†

nxmmxn

rmxrmxrrm

rmrxrxr††

nxmmxn

I

00

0

mxn

†

nxm

rnxrnxrrn

rnrxrxr†

mxn

†

nxm

I

00

0

Using the definition of the Pseudoinverse we can see that

54

SOLO Matrices



†

nxm

rmxrnxrrn

r

rmrx

rmxrnxrrn

r

rmrx

rnxrnxrrn

rnrxrxr†

nxmmxn

†

nxm

I

00

0

0

0

00

0

0

0

00

01

1

1

1

1

1

mxn

rnxrmxrrm

r

rnrx

rnxrmxrrm

r

rnrx

rmxrmxrrm

rmrxrxr

mxn

†

nxmmxn

I

00

0

0

0

00

0

0

0

00

0

11

†

nxmmxn

H†H††Def†

†H††H†H††

nxmmxn AAUUUUUUUVVUAA

mxn

†

nxm

H†H††Def†

†H††HH††

mxn

†

nxm AAVVVVVVVUUVAA

Also:

Let check the same operations for Matrix A †

mxn

HH†HH†H

mxn

†

nxmmxn AVUVUVUUVVUAAA

†

nxm

H†H††H†HH††

nxmmxn

†

nxm AUVUVUVVUUVAAA

55

SOLO Matrices


Properties of Moore-Penrose Pseudoinverse Matrix, A † - Summary)

†††nxmmxnnxmmxn AAAA

mxn

†

nxm

†

mxn

†

nxm AAAA

mxnmxn

†

nxmmxn AAAA

†

nxm

†

nxmmxn

†

nxm AAAA

1

2

3

4

Table of Contents

56

SOLO Matrices



bPbAAxAp †

R

11

HAR

AN

HAN

AR

Rx

Adomx

Acodomb


span by VA2H


H



b

bAx †

R

R

†

xA

bAAp

RxAp

P1 is a projection matrix because

11

1

2

1

PUUUVVUAAP

PAAAAAAPHH†HH†HH†H

†††

P1=A A† projects into column space of A, R (A)b

H†H†H† UUUVVUAAP :1

57

SOLO Matrices



2 HAR

AN

HAN

AR

Rx

Adomx

Acodomb


span by VA2H


H



b

bAx †

R

R

†

xA

bAAp

RxAp

bAAIpb †

0 pbA†

pAx †

R

P2=(I - A A † ) is a projection matrix because

21112

2

2

2

PPIPIPIAAIP

PAAAAAAAAIAAIAAIP

HHH†H

†

A

†††††

Because , is theprojection of into .

HANAR pb

HANb

We can see, also, that:

H†H†

†

UIUUUI

AAIPIP

12 :

00

bbAAAAbAAIApbA

†A

†††††† pAbAx ††

R

bPbAAIpb †

2

58

SOLO Matrices



3 NR

N

H

R

NR xxAofspaceNullANx

AofspaceRowARxxxx

xPxAAxAApAbAx ††††

R

3

P3=A †A is a projection matrix of in R (AH)x

33

3

2

3

PVVVUUVAAP

PAAAAAAAAAAP

HH†HHH†H†H

†

A

††††

†

4 xPxAAIxAAxxxx ††

RN

4

P4=I-A †A is a projection matrix of in N (A)x

43333

4

2

33

2

3

2

4

3

2

PPIPIPIP

PPPIPIP

HHHH

P

HAR

AN

HAN

AR

Adomx

Acodomb


span by VA2H


H



b

bAx †

R

R

†

xA

bAAp

RxAp

bAAIpb †

0 pbA†

pAx †

R

xAAx †

R

HAR

AN

HAN

AR

Adomx

Acodomb


span by VA2H


H



b

bAx †

R

R

†

xA

bAAp

RxAp

bAAIpb †

0 pbA†

pAx †

R

xAAx †

R

xAAIx †

N

0NxA

59

SOLO Matrices


Description of Projections Related to Moore-Penrose Pseudoinverse (Summary)

3 H††††

R ARxPxAAxAApAbAx

3

4

HAR

AN

HAN

AR

Adomx

Acodomb


span by VA2H


H



b

bAx †

R

R

†

xA

bAAp

RxAp

bAAIpb †

0 pbA†

pAx †

R

xAAx †

R

xAAIx †

N

0NxA

ANxPxAAIxAAxxxx ††

RN

4

H† ANbPbAAIpb

22

ARbPbAAxAp †

R

11

H†† UUAAP :1

†AAIPIP 12 :H†† VVAAP :3

AAIPIP † 34 :

Table of Contents

60

SOLO Matrices


Particular case (1) r = n ≤ m:

H

A

xnnm

A

AA

H

AAAmxn nxn

nxn

nmmxmxnnxnmxnmxmVUUVUA

0

1

21

(a) rank (Amxn) = n or

(b) columns of Amxn are linear independent or

(c) N (Amxn) = 0 or

(d) AnxmHAmxn is nonsingular

This is equivalent to:

where 0,,, 21211 nnA diagnxn

H

AA

H

AA

H

AAH

A

H

A

AA

nmxnm

nxn

AAH

A

H

A

A

H

A UUUUUUU

UUU

I

IUU

U

UUU

2211

2

1

2121

2

1

0

0

H

AAAH

A

H

A

nmnxAA

†

nxm nxmnxnnxn

xmnm

nxm

nxnnxnUV

U

UVA 1

1

1

2

11

1 0:

HAR

0AN HAN

AR

x

b


H



bAAxAp †

pAx †

bAAIpb

xAp

bAx †

0 pbA†

61

SOLO Matrices


Particular case (1) r = n ≤ n: (continue – 1)

H

AAAH

A

H

A

nmnxAA

†

nxm nxmnxnnxn

xmnm

nxm

nxnnxnUV

U

UVA 1

1

1

2

11

1 0:

H

A

xnnm

A

AA

H

AAAmxn nxn

nxn


0

1

21

H

AAA

H

A

A

AAH

A

H

A

AA

H VVVUUU

UVAA 2

1

1

21

2

1

1 00

H

AAA

H VVAA 2

1

1

†H

AAA

H

AAA

H

AAA

HH AUVUVVVAAA

1

1

111

2

1

1

or H

nxmmxn

H

nxm

†

nxm AAAA1

HAR

0AN HAN

AR

x

b


H



bAAxAp †

pAx †

bAAIpb

xAp

bAx †

0 pbA†

We have only one solution that minimize

11 mxnxmxn bxAd

x

and is given by:

1

1

11 mx

H

nxmmxn

H

nxmmx

†

nxmnx bAAAbAx Table of Contents

62

SOLO Matrices



(a) rank (Amxn) = m or

(b) rows of Amxn are linear independent or

(c) N (AnxmH) = 0 or

(d) AmxnAnxmH is nonsingular

This is equivalent to:

where 0,,, 21211 mmA diagmxm

H

AAA

H

A

xmmn

A

AnA

†

nxm mxmmxmnxmmxm

mxm

mnnxmxrUVUVVA 1

11

1

1

210

:

H

AAAH

A

H

A

mnmxAA

H

AAAmxn mxnmxmmxm

xnmn

mxn

mxmmxmnxnmxnmxmVU

V

VUVUA 11

2

1

1 0

H

AA

H

AA

H

AAH

A

H

A

AA

mnxmn

mxm

AAH

A

H

A

A

H

A VVVVVVV

VVV

I

IVV

V

VVV

2211

2

1

2121

2

1

0

0

HAR

AN

ARBAN H &0

AR

xAb

N

†

nxn xxAAI

R

† xxAA


span by VA2H


H Column Space of Aspan by UA1

bAx †

R

RxAb

b

0NxA

63

SOLO Matrices


Particular case (2) r = m ≤ n: (continue – 1)

H

AAA

H

A

xmmn

A

AnA

†

nxm mxmmxmnxmmxm

mxm

mnnxmxrUVUVVA 1

11

1

1

210

:

H

AAAH

A

H

A

mnmxAA

H

AAAmxn mxnmxmmxm

xnmn

mxn

mxmmxmnxnmxnmxmVU

V

VUVUA 11

2

1

1 0

H

AAA

H

AA

I

A

H

AAA

H UUUVVUAAm

2

11111 H

AAA

H UUAA 2

1

1

†H

AAA

H

AAA

H

AAA

HH AUVUUUVAAA 1

11

2

111

1

or 1 HH† AAAA HAR

AN

ARBAN H &0

AR

xAb

N

†

nxn xxAAI

R

† xxAA


span by VA2H



bAx †

R

RxAb

b

0NxA

We have an infinite number of solutions that minimize

11 mxnxmxn bxAd

bAAAbAx HH†

R

1

The solution that minimizesthe norm is given by:

Rx

Rx

Table of Contents

64

SOLO Matrices


X - nxp unknowns with mxp equations

mxpnxpmxn BXA




HAR

AN

HAN

ARBXA

11 yAx H

AdomX

AcodomY

1nxy


span by VA2H


H



B

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

H

AA

H

AA

H

AAH

A

H

A

AA

rmxrm

rxr

AAH

A

H

A

A

H

A UUUUUUU

UUU

I

IUU

U

UUU

2211

2

1

2121

2

1

0

0

H

AA

H

AA

H

AAH

A

H

A

AA

rnxrn

rxr

AAH

A

H

A

A

H

A VVVVVVV

VVV

I

IVV

V

VVV

2211

2

1

2121

2

1

0

0

65

SOLO Matrices


Let multiply byand using:

mxpnxpmxn BXA

H

H

U

U

2

1

we obtain:

BU

BUX

V

VUU

U

UH

A

H

A

H

A

H

AA

I

AAH

A

H

A

m

2

1

2

11

21

2

1

00

0

or:

BU

BUX

V

VH

A

H

A

H

A

H

AA

2

1

2

11

00

0

or:

xprmmxp

H

A BUxmrm

02

(m-r)xp - constraints equivalent to condition Bmxp (Amxn)

mxp

H

Anxp

H

AA BUXVrxmrxnrxr 111

rxp - independent equationsnxp – unknownssince r ≤ n → Eq. ≤ Unknown

HAR

AN

HAN

ARBXA

11 yAx H

AdomX

AcodomY


span by VA2H


H



B

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

66

xprmmxp

H

A BUxmrm

02

(m-r)xp - constraints equivalent to condition Bmxp (Amxn)

SOLO Matrices


mxp

H

Anxp

H


rxp - independent equationsnxp – unknownssince r ≤ n → Eq. ≤ Unknown

This equation is a Necessary and Sufficient Condition for any solutions of equationAmxn Xnxp = Bmxp. It is equivalent to Bmxp (Amxn) or Bmxp N (AT) = . If this condition is fulfilled, then from we havenxp unknowns ≥ rxp independent equations, that means (n-r)xp degrees of freedom.

mxp

H

Anxp

H


mxp

H

AAnxp

H

A BUXVrxmrxrrxn 1

1

11

Since VA1

T VA1=Ir & VA1T VA2 = 0 the

General Solution of Amxn Xnxp = Bmxp is:

AN

xprnA

AR

mxp

H

AAAnxp YVBUVXrnnx

T

rxmrxrnxr

21

1

11

where Y(n-r)xp is any (n-r)xp matrix, i.e. weused all (n-r)xp degrees of freedom.

HAR

AN

HAN

ARBXA AdomX

AcodomY


span by VA2H


H



BB has to be in thecolumn space of A

ANBorARB

67

SOLO Matrices


Check:

mxpmxp

xmrm

H

A

H

A

AAmxpxmrm

H

A

mxp

H

A

AA

xprm

mxp

H

A

AA

xprn

mxp

H

AA

rnxrmxrrm

rnrxA

AA

xprn

I

A

H

Amxp

H

AAA

H

A

xprnA

H

Amxp

H

AA

I

A

H

A

rnxrmxrrm

rnrxA

AA

xprnAmxp

H

AAAH

A

H

A

rnxrmxrrm

rnrxA

AAnxp

H

AAAnxpmxn

BBU

UUUBU

BUUU

BUUU

Y

BUUU

YVVBUVV

YVVBUVVUU

YVBUVV

VUUXVUXA

rxm

rmmxmxr

rxm

rmmxmxr

rxm

rmmxmxr

rxmrxrrxr

rmmxmxr

rnnxxnrnrxmrxrnxrxnrn

rnnxrxnrxmrxrnxrrxnrxr

rmmxmxr

rnnxrxmrxrnxr

xnrn

rxnrxr

rmmxmxrnxnmxnmxm

2

1

21

0

2

1

21

1

21

1

1

11

21

221

1

1

0

12

0

211

1

1111

21

21

1

11

2

11

21

000

0

00

0

00

0

68

SOLO Matrices

where r is such that:


Algorithm to solve Amxn Xnxp = Bmxp:

(1) Compute s.v.d. of Amxn and partition according to:

0,,, 21211 rrA diagrxr

(2) Check if: xprmmxp

H

A BUxmrm

02

(3) If (2) is not true → no solution for (1)

any

xprnAmxp

H

AAAnxp YVBUVXrnnxrxmrxrnxr

21

1

11

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

If (2) is true → (n-r)xp solutions:

69

SOLO Matrices General Solution of Amxn Xnxp = Bmxp

Moore-Penrose Pseudoinverse of A:

H

AAAH

A

H

A

rmxrnxrrn

rmrxA

AnA

†

nxm rxmrxrnxr

xmrm

rxmrxr

rnnxmxrUV

U

UVVA 1

1

11

2

11

1

2100

0:

then

H

AAH

A

H

A

rmxrnxrrn

rmrxA

I

AAH

A

H

A

rnxrmxrrm

rnrxA

AA

†

mxn rxnnxr

xmrm

rxmrxr

nxn

rnnxnxr

xnrn

rxnrxr

rmmxmxrnxmUU

U

UVV

V

VUUAA 11

2

11

1

21

2

11

2100

0

00

0

H

AA

H

AA

H

AA

H

AA

†

nxmmxnmxm xmrmrmmxrxmmxrxmrmrmmxrxmmxrUUUUUUUUAAI

22112211

also

H

AAH

A

H

A

rnxrmxrrm

rnrxA

I

AAH

A

H

A

rmxrnxrrn

rmrxA

AAnxm

†

mxn rxnnxr

xnrn

rxnrxr

mxm

rmmxmxr

xmrm

rxmrxr

rnnxnxrVV

V

VUU

U

UVVAA 11

2

11

21

2

11

1

21 00

0

00

0:

H

AA

H

AA

H

AA

H

AAmxn

†

nxmnxn xnrnrmnxrxnnxrxnrnrmnxrxnnxrVVVVVVVVAAI

22112211

70

SOLO Matrices


Moore-Penrose Pseudoinverse of A (continue - ):

Define also Znxp such that: nxp

H

Axprn ZVYxrrn

2:

any

xprnAmxp

H

AAAnxp YVBUVXrnnxrxmrxrnxr

21

1

11

Since, if xprmmxp

H

A BUxmrm

02

The solution of Amxn Xnxp = Bmxp is

H

AA

H

AA

H

AA

H

AAmxn

†


22112211

H

AAAH

A

H

A

rmxrnxrrn

rmrxA

AnA

†

nxm rxmrxrnxr

xmrm

rxmrxr

rnnxmxrUV

U

UVVA 1

1

11

2

11

1

2100

0:

Therefore: any

nxpmxn

†

nxmnxnmxp

†

nxmnxp ZAAIBAX

Note: By writing the solution this way we lose the fact that we have only (n-r)xp different solutions as we have seen.

Check:

xprn

xnrnrnnxrxmrxrnxr

Y

nxp

H

AAmxp

H

AAAnxp ZVVBUVX

221

1

11

71

SOLO Matrices



any

nxpmxn

†

nxmnxnmxp

†

nxm

any

xprnAmxp

H

AAAnxp ZAAIBAYVBUVXrnnxrxmrxrnxr

21

1

11

xprmmxp

H

AA

ANonBofprojection

mxp

H

BUU

BAAI

orANB

orARB

xmrmrmmx

T

0

0

22

Solutions exists iff:

HAR

AN HAN

AR

BXA

YVZAAI A

†

nxn 2

BA†


span by VA2H




B

BA†Z

i.e. the projection (Imxm – Amxn Anxm †) of B on N (AH) is zero.

mxpmxp

H

AAmxp

†

nxmmxnmxm BUUBAAImnrmrmmx

0

0

22

Table of Contents

72

SOLO Matrices



solutions always exist

HAR

AN

ARBAN H &0

AR

BXA

YVZAAI A

†

nxn 2

BA†


span by VA2H



B

BA†Z

Since

H

A

H

A

mnmxAA

H

AAAmxn

xnmn

rxn

mxmmxmnxnmxnmxm V

VUVUA

2

1

1 0

xprmmxp

H

ArmmxA BUUxmrmrmmx

00 22

nxp unknowns ≥ mxp equations, meaning (n-m)xp degrees of freedom

any

nxpmxn

†

nxmnxnmxp

†

nxm

any

xprnAmxp

H

AAAnxp

ZAAIBA

YVBUVXrnnxrxmrxrnxr

21

1

11

11

11

1

1

210

:

H

nxmmxn

H

nxm

H

AAA

H

A

xmmn

A

AnA

†

nxm AAAUVUVVAmxmrxrnxrmxm

mxm

mnnxnxm

mxm

H

A

xmmn

A

AAH

A

H

A

mnmxAA

†

nxmmxn IUVVV

VUAA

mxm

mxm

mnnxnxm

xnmn

mxn

mxmmxm

0

01

1

21

2

1

Table of Contents

73

SOLO Matrices


Particular case (2) r = n ≤ n: mxp equations ≥ nxp unknowns, meaning (n-m)xp constraints

Only if solutions exist. xprmmxp

H

A BUxmrm

02

In this case we have nxp unknowns and mxp equations - (m-p)xp constraints = nxp independent equations, i.e. a unique solution:

mxpnxmmxp

H

AAAnxp BABUVXnxmnxnnxn

1

1

11

H

A

xnnm

A

AA

H

AAAmxn nxn

nxn


0

1

21

H

AAAH

A

H

A

nmnxAA

†

nxm nxmnxnnxn

xmnm

nxm

nxnnxnUV

U

UVA 1

1

2

11 0:

nxn

H

A

xnnm

A

AAH

A

H

A

nmnxAAmxn

†

nxm IVUUU

UVAA

nxn

nxn

nmmxmxn

xmnm

nxm

nxnnxn

0

01

21

2

11

HAR

0AN HAN

ARBXA

BA†1nxy


HColumn Space of A

span by UA1


BB has to be in thecolumn space of A

ANBorARB

Table of Contents

74

SOLO Matrices


Y - pxm unknowns with pxn equations

pxnmxnpxm CAY




H

AA

H

AA

H

AAH

A

H

A

AA

rmxrm

rxr

AAH

A

H

A

A

H

A UUUUUUU

UUU

I

IUU

U

UUU

2211

2

1

2121

2

1

0

0

H

AA

H

AA

H

AAH

A

H

A

AA

rnxrn

rxr

AAH

A

H

A

A

H

A VVVVVVV

VVV

I

IVV

V

VVV

2211

2

1

2121

2

1

0

0

HAR

AN HAN

ARCAY

11 yAx H

CY


span by VA2H


H



H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

75

SOLO Matrices

we obtain: 2121

2

11

21 00

0AAAAH

A

H

AA

AA VCVCVVV

VUUY

or:

rnpxApxn rnnxVC

02

px(n-r) - constraints equivalent to condition Cpxn (Amxn

H)

nxrrxrmxr ApxnAApxm VCUY 111 pxr - independent equationspxm – unknownssince r ≤ m → Eq. ≤ Unknown

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0

General Solution of YpxmAmxn = Cpxn HAR

AN HAN

ARCAY

CY


span by VA2T


T



C has to be in theRow Space of A

HAC

ANC

orARC HLet post-multiply byand using:

21 VVpxnmxnpxm CAY

or: 21

1

21 00

0AA

A

AA VCVCUUY

76

SOLO Matrices

Since UA1T UA1=Ir & UA1

T UA2 = 0 the

General Solution of YpxmAmxn = Cpxn is:

where Xpx(m-r) is any px(m-r) matrix, i.e. weused all px(m-r) degrees of freedom.


nxrrxrmxr ApxnAApxm VCUY 111 pxr - independent equationspxm – unknownssince r ≤ m → Eq. ≤ Unknown

This equation is a Necessary and Sufficient Condition for any solutions of equationYpxmAmxn = Cpxn. It is equivalent to Cpxn (Amxn) or Cpxn N (AT) = . If this condition is fulfilled, then from we havenxp unknowns ≥ rxp independent equations, that means (n-r)xp degrees of freedom.

H

ApxnAApxm nxrrxrmxrVCUY 111

1

111

rxrnxrmxr AApxnApxm VCUY

px(n-r) - constraints equivalent to condition Cpxn (Amxn

H) rnpxApxn rnnxVC

02

H

A

any

rmpx

H

AAApxnpxm xmrmrxmrxrnxrUXUVCY

21

1

11

HAR

AN HAN

ARCAY

CY


span by VA2H


H




HAC

ANC

orARC H

HAC

77

SOLO Matrices

Check:

pxnH

A

H

A

AApxnH

A

H

A

ApxnApxn

H

A

H

A

rnpxApxnH

A

H

A

rnxrmxrrm

rnrxA

rmpxAApxn

H

A

H

A

rnxrmxrrm

rnrxA

I

A

H

ArmpxA

H

AAApxnA

H

Armpx

I

A

H

AAApxn

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

Armpx

H

AAApxnmxnpxm

CV

VVVC

V

VVCVC

V

VVC

V

VXVC

V

V

UUXUUVCUUXUUVC

V

VUUUXUVCAY

xnrn

rxn

rnpxnxr

xnrn

rxn

rnpx

rnpxnxr

xnrn

rxn

nxr

xnrn

rxnrxr

rxrnxr

xnrn

rxnrxr

rmmxxmrmrmmxrxmrxrnxrmxrxmrmmxrrxmrxrnxr

xnrn

rxnrxr

rmmxmxrxmrmrxmrxrnxr

2

1

21

2

1

0

21

2

1

1

2

111

11

2

11

22

0

21

1

11

0

1211

1

11

2

11

2121

1

11

000

0

00

0

00

0


78

SOLO Matrices

where r is such that:

Algorithm to solve YpxmAmxn = Cpxn:

(1) Compute s.v.d. of Amxn and partition according to:

0,,, 21211 rrA diagrxr

(2) Check if:

(3) If (2) is not true → no solution for (1)

H

A

H

A

rnxrmxrrm

rnrxA

AA

H

AAAmxn

xnrn

rxnrxr

rmmxmxrnxnmxnmxm V

VUUVUA

2

11

21 00

0


rnpxApxn rnnxVC

02

H

A

any

rmpx

H


21

1

11

If (2) is true → px(m-r) solutions:

79

SOLO Matrices


Define also Wpxm such that: rmmxApxmrmpx UWX

2:

H

AA

H

AA

H

AA

H

AAmxn

†


22112211

H

AAAH

A

H

A

rmxrnxrrn

rmrxA

AnA

†

nxm rxmrxrnxr

xmrm

rxmrxr

rnnxmxrUV

U

UVVA 1

1

11

2

11

1

2100

0:

Therefore: †

nxmmxnmxm

any

pxm

†

nxmpxnpxm AAIWACY

Note: By writing the solution this way we lose the fact that we have only px(m-r) different solutions as we have seen.

If rnpxApxn rnnx

VC

02


the solution of YpxmAmxn = Cpxn is

H

A

any

rmpx

H


21

1

11

Check:

H

A

X

Apxm

H

AAApxnpxm xmrm

rmpx

rmmxrxmrxrnxrUUWUVCY

221

1

11

80

SOLO Matrices


Solutions exists iff:i.e. the projection (Inxn – Anxm

† Amxn) of C on N (A) is zero.


†

nxmmxnmxm

any

pxm

†

nxmpxn

H

A

any

rmpx

H

AAApxnpxm

AAIWAC

UXUVCYxmrmrxmrxrnxr

21

1

11

pxn

H

AApxnmxn

†

nxmnxnpxn xnrn

rnpx

rnnxVVCAAIC 02

0

2

HAR

AN HAN

ARCAY

CpxmY


span by VA2H


H




HAC orANC

orARC H

pxn

H

AApxn

ANonprojection

mxnnxmnxnpxn xnrn

rnpx

rnnxVVCAAIC 02

0

2

H

Armpx

ANonWanyofprojection

nxmmxnmxmpxm xrrm

Hpxm

UXAAIW

2

pxmW

HAC

pxn

H

AApxn

ANonprojection

mxn

†

nxmnxnpxn

H

xnrn

rnpx

rnnxVVC

AAIC

orANC

orARC

02

0

2

†††

Table of Contents

81

SOLO Matrices


H

A

H

A

mnmxAA

H

AAAmxn

xnmn

rxn

mxmmxmnxnmxnmxm V

VUVUA

2

1

1 0

11

11

1

1

210

:

H

nxmmxn

H

nxm

H

AAA

H

A

xmmn

A

AnA

†

nxm AAAUVUVVAmxmrxrnxrmxm

mxm

mnnxnxm

mxm

H

A

xmmn

A

AAH

A

H

A

mnmxAA

†

nxmmxn IUVVV

VUAA

mxm

mxm

mnnxnxm

xnmn

mxn

mxmmxm

0

01

1

21

2

1


Only if solutions exist. rnpxApxn rnnx

VC

02

In this case we have pxm unknowns and pxn equations – px(n-m) constraints = pxm independent equations, i.e. a unique solution:

†

nxmpxn

H

AAApxnpxm

AC

UVCYrxmrxrnxr

1

1

11

HAR

AN 0HAN

ARCAY

C


span by VA2H


H



HAC

pxn

H

AApxnmxn

†

nxmnxnpxn

H

xnrn

rnpx

rnnxVVCAAIC

orANC

orARC

02

0

2

Table of Contents

82

SOLO Matrices

Particular case (2) r = n ≤ n: pxn equations ≥ pxm unknowns, meaning px(m-n) constraints

H

A

xnnm

A

AA

H

AAAmxn nxn

nxn


0

1

21

H

AAAH

A

H

A

nmnxAA

†

nxm nxmnxnnxn

xmnm

nxm

nxnnxnUV

U

UVA 1

1

2

11 0:

nxn

H

A

xnnm

A

AAH

A

H

A

nmnxAAmxn

†

nxm IVUUU

UVAA

nxn

nxn

nmmxmxn

xmnm

nxm

nxnnxn

0

01

21

2

11

Since solutions always exist rnpxApxn rnnx

VC

02

pxm unknowns ≥ pxn equations, meaning px(m-n) degrees of freedom

HAR

0AN HAN

ARCAY

CY


H




HAC

CAAC †

H

AUXAAIW 2

pxn

H

AApxnmxn

†

nxmnxnpxn

H

xnrn

rnpx

rnnxVVCAAIC

orANC

orARC

02

0

2

†

nxmmxnmxm

any

pxm

†

nxmpxn

H

A

any

rmpx

H

AAApxnpxm

AAIWAC

UXUVCYxmrmrxmrxrnxr

21

1

11


Table of Contents

83

SOLO Matrices Inverse of Partitioned Matrices

11111

111111

mnnnnmmmnmmmmnnnnmmm

mnnnnmmmmnnnnmmmmnnn

mmnm

mnnn

BADCDCBADC

BADCBADCBA

CD

BA

if and exist. 1nnA 1

nnC

Let find the inverse of such that:

mmnm

mnnn

PN

ML

mmnm

mnnn

CD

BA

mmnm

mnnn

mmnm

mnnn

mmnm

mnnn

I

I

PN

ML

CD

BA

0

0

Proof:

nnnmmnnnnn INBLA 1 nnnnnmmmmnnn ILDCBA

1

2 nmnmmmnnnm NCLD 0nnnmmmnm LDCN

1

3mnmmmnmnnn PBMA 0

mmmnnnmn PBAM

1

4mmmmmmmnnm IPCMD mmmmmnnnnmmm IPBADC

1

84


11111

111111



mmnm

mnnn

BADCDCBADC

BADCBADCBA

CD

BA


nnC

Let find the inverse of such that:

mmnm

mnnn

PN

ML

mmnm

mnnn

CD

BA

mmnm

mnnn

mmnm

mnnn

mmnm

mnnn

I

I

PN

ML

CD

BA

0

0

Proof (continue – 1):

1 nnnnnmmmmnnn ILDCBA

1

2 nnnmmmnm LDCN

1

3

4

mmmnnnmn PBAM

1

mmmmmnnnnmmm IPBADC

1

11

nmmmmnnnnn DCBAL

111

nmmmmnnnnmmmnm DCBADCN

11

mnnnnmmmmm BADCP

111

mnnnnmmmmnnnmn BADCBAM

q.e.d.

85


mmnm

mnnn

mmnm

mnnn

mmnm

mnnn

I

I

CD

BA

PN

ML

0

0From:

1

mmmnnmmmmm CBNIPmmmmmmmnnm ICPBN we get:

11

mnnnnmmmmm BADCPSubstitute:

and: 111

nmmmmnnnnmmmnm DCBADCN

1111111

mmmnnmmmmnnnnmmmmmmnnnnmmm CBDCBADCCBADC

to obtain:

By inter-changing , in this identity, we obtain:nmmnnnmm DBAC ,

1111111

nnnmmnnnnmmmmnnnnnnmmmmnnn ADBADCBAADCBA

86

SOLO

Pre-multiplying this identity by we obtain

Matrices Inverse of Partitioned Matrices

Let prove the identity:

111111

mmmnnmmmmnnnmnnnnmmmmnnn CBDCBABADCBA

1111

mnnnnmmmmnnnnmmmmnnn BADCBADCBA

Proof:

1111

mnnnnmmmmnnnnmmmmnmn BADCBADCBB

11111

mmmnmnnnnmmmmnnnnmmmmmmn CBBADCBADCCB

11

nmmmmnnn DCBA

111

111111

mmmnnmmmmnnn

mnnnnmmmmnnnnmmmmnnnnmmmmnnn

CBDCBA

BADCBADCBADCBA

q.e.d.

111111

nmmmmnnnnmmmnnnmmnnnnmmm DCBADCADBADC

By inter-changing ,in the first identity, we obtain:nmmnnnmm DBAC ,

111111

nnnmmnnnnmmmnmmmmnnnnmmm ADBADCDCBADC

q.e.d.

87


By using the identities:

111111

nmmmmnnnnmmmnnnmmnnnnmmm DCBADCADBADC

We obtain:

11111

111111



mmnm

mnnn

BADCDCBADC

BADCBADCBA

CD

BA


nnC

11111

111111111

mnnnnmmmnnnmmnnnnmmm

mnnnnmmmmnnnnnnmmnnnnmmmmnnnnn

mmnm

mnnn

BADCADBADC

BADCBAADBADCBAA

CD

BA

1111111

nnnmmnnnnmmmmnnnnnnmmmmnnn ADBADCBAADCBA

88


If and exist, performing the computation M-1M, we can prove: 1nnA 1

nnC

111

1100

mmnnnmmm

mnnn

mmnm

mnnn

CADC

A

CD

A1

2

1

1111

00mmnm

mmmnnnnn

mmnm

mnnn

C

CBAA

C

BA

3

1

11

0

0

0

0

mmnm

mnnn

mmnm

mnnn

C

A

C

A

4 If and : T

nnnn AA T

nnnn CC

11111

111111

mnnnnmT

mmnmT

mmmnnnnmT

mm

mnnnnmT

mmmnnnnmT

mmmnnn

mmnmT

mnnn

BABCBCBABC

BABCBABCBA

CB

BA

Because this is a symmetric matrix

111111

nnnm

Tmnnnnm

Tmmnm

Tmmmnnnnm

Tmm ABBABCBCBABC

Also: 1111111

mmmnnmT

mmmnnnnmT

mmmmmnnnnmT

mm CBBCBABCCBABC

89


If m=n and and also exist: 1

nnB1

nnD5

111111

nnnnnnnnnnnnnnnnnnnnnnnn ABBADCBADCBA

111111

nnnnnnnnnnnnnnnnnnnnnnnn CDDCBADCBADC

also

we obtain

1111

11111

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnn

nnnn

BADCCDAB

ABCDDCBA

CD

BA

Table of Contents

90

SOLO Matrices


1111111

mmmnnmmmmnnnnmmmmmmnnnnmmm CBDCBADCCBADC1

Proof:

1111111


In the identity:

substitute by .1

nnA nnA

Substitute by and by in (1).1

nnAnnA mmC

1

mmC2


1111

Substitute in (1) nnnn IA 3

111111

mmmnnmmmmnnnnmmmmmmnnmmm CBDCBIDCCBDC

Substitute in (2) nnnn IA 4

mmmnnmmmmnnnnmmmmmmnnmmm CBDCBIDCCBDC

111

91

SOLO Matrices


5Substitute in (1), (2), (3), (4) by . (We don’t assume symmetric and )nmD nm

TB nnA mmC

1111111

mmmnnm

Tmmmnnnnm

Tmmmmmnnnnm

Tmm CBBCBABCCBABC

mmmnnmT

mmmnnnnmT

mmmmmnnnnmT

mm CBBCBABCCBABC

1111

111111

mmmnnm

Tmmmnnnnm

Tmmmmmnnm

Tmm CBBCBIBCCBBC

mmmnnmT

mmmnnnnmT

mmmmmnnmT

mm CBBCBIBCCBBC

111

From this we get:

6Substitute in (3) mmmm IC

mnnmmnnnnmmmmnnmmm BDBIDIBDI

11

mnnmmnnmmmmnnmmmmnnmmmmnnmmm

mnnmmmmnnmmnnmmmmnnmmmmnnmmm

mnnmmmmmmnnmmnnnnm

BDBDIBDIBDIBDI

BDIBDBDIBDIBDI

BDIIBDBID

111

111

11

92

SOLO Matrices


7Substitute in the identity

111111

mmmnnmmmmnnnmnnnnmmmmnnn CBDCBABADCBA

nnnn IA and mmmm IC to obtain:

mnnmmnnnmnnmmmmn BDBIBDIB

11

Pre-multiplying this by we get (6).nmD

By using a similar path with the identity

111111


nnnn IA and mmmm IC to obtain:with

nmmnnmmmnmmnnnnm DBDIDBID

11

Post-multiplying this by we get (6).mnB

93

SOLO Matrices

By matrix manipulation we obtain:


8In the identity

111111

mmmnnmmmmnnnmnnnnmmmmnnn CBDCBABADCBApre-multiplying Anxn by and post-multiplying by Cmxm we get:

mnnmmmmnnnnnmmmnnnnmmmmn BDCBAACBADCB

1111

mnnnnmmmmnnnmnnnnmmmmmmn BADCBIBADCIB

111111

Use now the identity

111111


Pre-multiplying by Cmxm and post-multiplying by Anxn we get:

nmmnnnnmmmmmnnnmmmmnnnnm DBADCCADCBAD

1111

By matrix manipulation we obtain

nmmmmnnnnmmmnmmmmnnnnnnm DCBADIDCBAID

111111

Table of Contents

94

SOLO Matrices

Matrix Schwarz Inequality

QPPPQPQQ TTTTT 1

Table of Contents

Hermann Amandus Schwarz

1843 - 1921yxyx ,

Let x, y be the elements of an Inner Product space X, than :

This is the Schwarz Inequality.

Let Pmxn and Qmxl be two matrices such that PTP is nonsingular, then:

CxxQPPPQPxxQQx TTTTTTT 1i.e.,:

Furthermore equality holds if and only if exists a matrix Snxl such that Q = P S.

Proof:

Start from the inequality: and choose 0 SPQSPQ T QPPPS TT 1

01

1111

QPPPQPQQ

QPPPPPPPPQQPPPPQQPPPPQQQ

SPPSQPSSPQQQSPQSPQ

TTTTT

TTTTTTTTTTTT

TTTTTTT

The inequality becomes equality if and only if : that is equivalent with

0 SPQSPQ T

SPQ

95

SOLO Matrices

Trace of a Square Matrix The trace of a square matrix is defined as T

nn

n

iiinn AtraceaAtrace

1

:

q.e.d.

ABtraceBAtrace 1

Proof:

n

i

n

jjiij baBAtrace

1 1

BAtracebaabABtracen

i

n

jjiij

n

j

n

iijji

1 11 1

ABtraceBAtraceBAtraceABtraceABtraceBAtrace TTTT111

2

Proof:

ABtraceBAtracebabaBAtracen

i

n

jjiij

n

i

n

jijij

T

1 11 1

Tn

j

n

iijij

T BAtraceabABtrace

1 1q.e.d.

96

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

3

Proof:

q.e.d.

n

ii APAPtraceAtrace

1

1

where P is the eigenvector matrix of A related to the eigenvalue matrix Λ of A by

n

PPPA

0

01

AtraceAPPtracePAPtrace 11

1

n

PPPA

0

01

n

PAP

0

01

1

n

iitracePAPtace

1

1

97

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

Proof:

q.e.d.

Definition

4 AtraceA ee det

AtraceA eeePeP

PePPePe

n

i

i

1detdetdet

det

1detdetdetdetdet 11

If aij are the coefficients of the matrix Anxn and z is a scalar function of aij, i.e.:

njiazz ij ,,1,

then is the matrix nxn whose coefficients i,j areA

z

njia

z

A

z

ijij

,,1,:

(see Gelb “Applied Optimal Estimation”, pg.23)

98

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

Proof:

q.e.d.

5

A

AtraceI

A

Atrace T

n

1

ji

jia

aA

Atraceij

n

iii

ijij1

0

1

6 nmmnTTT RBRCCBBC

A

BCAtrace

A

ABCtrace

1

Proof:

ijTji

m

ppijp

ik

jl

n

l

m

p

n

kklpklp

ijij

BCBCbcabcaA

ABCtrace

11 1 1

q.e.d.

7 If A, B, C Rnxn,i.e. square matrices, then

TTT CBBCA

BCAtrace

A

CABtrace

A

ABCtrace

11

99

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

Proof:

q.e.d.

8 nmmn

TTT

RBRCBCA

ABCtrace

A

BCAtrace

A

ABCtrace

721

9

BC

A

BCAtrace

A

CABtrace

A

ABCtrace TTT 811

If A, B, C Rnxn,i.e. square matrices, then

10 TA

A

Atrace2

2

ijTjiji

n

l

n

mmllm

ijijij

Aaaaaaa

Atrace

A

Atrace2

1 1

22

11 1

kT

k

AkA

Atrace

Proof: 1111

kT

k

kTkTkT

k

k

AkAAAA

AAAtrace

A

Atrace

q.e.d.

100

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

Proof:

q.e.d.

12 TAA

eA

etrace

TAn

k

n

k

kT

n

kkkT

n

n

k

k

n

n

k

k

n

A

eAk

Ak

k

k

Atrace

Ak

Atrace

AA

etrace

1 0

11

00 !

1lim

!lim

!lim

!lim

13

TT

TTTTTTTTT

TTTTT

TTT

BACBAC

A

ACABtrace

A

BACAtrace

A

ABACtrace

A

CABAtrace

A

BACAtrace

A

CABAtrace

A

ACABtrace

A

BACAtrace

A

ABACtrace

111

21

11

TTTTTTT

BACBACCABBACA

ABACtrace

A

ABACtrace

A

ABACtrace

86

2

2

1

1Proof: q.e.d.

14

AA

AAtrace

A

AAtrace TT

213

101

SOLO Matrices


nn

n

iiinn AtraceaAtrace

1

:

Proof:

15 TTTTT ABBAABBAA

ABAtrace

Table of Contents

ijTTTTn

ljlli

n

kkijk

n

l

n

l

n

kklmklm

ijijij

ABBAbababaaaa

ABAtrace

A

ABAtrace

111 1 1

q.e.d.

16 TTTTTT CABBACA

ABACtrace

ijTTTTTTn

l

n

rlirljr

n

k

n

mmikmjk

n

l

n

rrljrli

n

k

n

mmikmjk

n

l

n

k

n

m

n

rrlmrkmlk

ijijij

CABBACcabbac

abcbacabacaa

ABACtrace

A

ABACtrace

1 11 1

1 11 11 1 1 1

Proof:

q.e.d.

102

SOLO

References [1] Pease, “Methods of Matrix Algebra” ,Mathematics in Science and Engineering,

Vol.16, Academic Press, 1965

Matrices

[2] S. Hermelin, “Robustness and Sensitivity Design of Linear Time-Invariant Systems” PhD Thesis, Stanford University, 1986

Table of Contents

April 11, 2023 103

SOLO

TechnionIsraeli Institute of Technology

1964 – 1968 BSc EE1968 – 1971 MSc EE

Israeli Air Force1970 – 1974

RAFAELIsraeli Armament Development Authority

1974 – 2013

Stanford University1983 – 1986 PhD AA

Matrices

104

SOLO

Derivatives of Matrices

Matrices

ljikij

kl

a

a

For vector forms

j

i

ijii

i

iy

x

y

x

y

x

y

x

y

x

y

x

:::

We have the following expressions:

HH

TT

XX

XX

XXtraceX

XXtraceXX

XXXX

YXYXYX

YXYXYX

XtraceXtrace

YXYX

XX

constAifA

1

1

111

detln

detdet

0

105

SOLO

Derivatives of Determinants

Matrices

x

YYtraceY

x

Y 1detdet

x

YY

x

YYtrace

x

YYtrace

x

YYtrace

x

xY

YtraceYx

xY

11

11

1det

det

General Form

106

SOLO


Matrices

11

1

detdetdet

detdet

detdet

TT

ijk

jkik

T

XBXAXBXAX

BXA

XXX

X

XXX

X

Linear Form

Square FormsIf X is Square and Invertible, then

TTT

XXAXX

XAX

det2

det

If X is Not Square but A is Symmetric, then

1det2

det

XAXXAXAXX

XAX TTT

If X is Not Square and A is Not Symmetric, then

11det

det

XAXXAXAXXAXAXX

XAX TTTTT

107

SOLO


Matrices

Tk

k

T

TT

TT

XXkX

X

XXX

X

XX

XX

XX

XX

detdet

22detln

2detln

2detln

1T1-

†

†

Nonlinear Form

108

SOLO

Derivatives of an Inverse

Matrices

111

Yx

YY

x

Y

From this it follows

T

T

T

TTTT

jlkiij

kl

AXAXX

AXtrace

XABXX

BXAtrace

XXX

X

XbaXX

bXa

XXX

X

111

111

111

1

111

detdet

109

SOLO

Derivatives of Matrices, Vectors and Scalar Forms

Matrices

First Order

ijnm

mjinmn

ijT

ijmn

njimmn

ij

ij

ij

TTTT

TTT

TT

TT

AJAX

AX

AJAX

AX

JX

X

aaX

aXa

X

aXa

abX

bXa

baX

bXa

ax

xa

x

ax

n

mJ mn

000

010

000

110

SOLO


Matrices

Second Order

jlikklijijijT

ij

T

ilkjklT

ljij

klT

TTT

T

TTTT

klkl

klmnmnkl

ij

JXBJJBXX

XBX

XBBXX

XBX

bxBCDdxDCBx

dxDCbxB

bccbXX

cXXb

XXXX

2

n

mJ mn

000

010

000

111

SOLO


Matrices

Second Order (continue)

TTT

TTTTT

TT

bcbXDDcbXDcbXX

bcXDcbXDX

cXDXb

xBBx

xBx

Assume W is symmetric

TT

T

T

T

TT

ssAxWsAxWsAxA

sAxWsAxWsAxx

sxWsxWsxs

sxWsxWsxx

sAxWAsAxWsAxs

2

2

2

2

2

112

SOLO


Matrices

Higher Order and Nonlinear

1

0

1n

rkl

rnijr

ij

kln

XJXX

X

1

0

11

11

0

n

r

TrnTnTrrTnTrnnTnT

Trnn

r

TTrnT

XbaXXXXbaXbXXaX

XbaXbXaX