vector and matrix derivation

7/24/2019 Vector and Matrix Derivation

1/4

APPENDIX D

VECTOR AND MATRIX

D

IFFERENTIAT

I0

N

Definition D .l

(Gradient) Let

f

x ) be a scalar finction

of

the elements of the vector

z XI . .

.

X N ) ~ .

hen, the gradient (v ec tor )

off z)

with respect

to

x is defined as

The transpose of the gradient

is

the column vec tor

DefinitionD.2

(Hessian matrix) Let f x ) be a twice continuously diferentiable scalar

function of the elements of the vector x XI X N ) ~ .hen, the Hessian matrix off x)

65

Parameter stimation

f o r

Scientists and ngineers

by Adriaan van den

Bos

Copyright

007

John W iley Sons Inc.


2/4

66

VECTOR ND M TRIX DIFFERENTI TION

with respect to

x

is defined as

Since, under the assumptions made, a x)/dx,dx,

a

x)/axqaxp,he Hessian

matrix is symmetric.

Dehition D3

Jacobian ma trix) Let f

x ) e

a

K

x

1 vectorfunction

o

he elements

o

he L x 1vector

x.

Then, the

K

x

L

Jacobian matrix

of f x )

with respect

to

x

s

defined as

The transpose

o

the Jacobian matrix is

Definition D.4 Let the elements

of

the M x

N

matrix

A

befunctions

o

the elements

xqo

a vector

x.

Then, the M x

N

matrix

aA/dx,

is defined as


3/4

67

and the matrix of second-order derivatives as

d2al l a2a lN

ax,ax, ax,ax,

~

Thus, the derivative of a matrix is the matrix

of

the derivatives.

Theorem D.1 Product dzferentiation rule fo r matrices) Let A and B be an x M and

an M x L matrix, respectively, and let C be the product matrix A

B.

Furthermore, suppose

that the elements

of A

and B arefunctions

of

the elements

xp f

a vector

x.

Then,

c

~

bB

B + A - - .

ax, a x p ax,

Proof By definition, the ( k ,C)-th elemen t

of

the matrix

C

is described by

m=

1

Then, the product rule for differentiation yields

and hence, by D.7),

dC d A d B

-B A -

ax, ax, ax,

D.10)

D.11)

D.12)

This com pletes the proof.

TheoremD 2 Let the

N

x N matrix A be nonsingular and let the elements of A befunctions

of

the elements

xq

f

a

vector

x.

Then, thefirs t-order and the second-order derivatives of

the inverse A-' with respect to the elements of

x

are equal to, respectively,

D.13)

and

2A-1 A- l (& A-1- d A 2 A

I

d A A - l f i ) A - l .

D.14)

ax,ax, ax ax ax axq ax ax

Proof Differentiating AA-' I where I is the identity matrix of order N, yields

where 0 is the N x N null matrix. Then,

D.15)

D.16)


4/4

68

VECTOR ND M TRIX DIFFERENTI TION

This

expression shows that

D.17)

Applying Theorem

D.

to this expression yields

Subsequently substituting the first order derivatives

D.

6 of A l in thisexpression shows

that

D.19)

2A-1 A- l d A

A - 1 - d A ___2 A a AA - l - ) 8 A A - l .

ax,axg axp

xg

axpaxq

axo axp

This completes the proof.

vector and matrix derivation

Documents