introduction to matrix algebra - appalachian state university · introduction to matrix algebra...
TRANSCRIPT
Introduction to Matrix Algebra
George H Olson, Ph. D.
Doctoral Program in Educational Leadership
Appalachian State University
September 2010
What is a matrix? Dimensions and order of a matrix. A p by q dimensioned matrix is a p (rows) by q (columns) array of numbers, or symbols, and will itself be represented by an upper-case, bold, italic symbol; e.g, A , β . For instance, a 2 by 3 matrix, A , can
be represented, as a symbolic array,
23
13
22
12
21
11
a
a
a
a
a
aA ,
or as an actual array of numbers
3
6
4
5
2
3A .
In either case, the order of A is said to be 2 3. On the other hand, the matrix,
434241
333231
232221
131211
ddd
ddd
ddd
ddd
D ,
is a 4 3 matrix. That is, its order is 4 3. A matrix having only one column or one row is called a vector and is represented by a lower-case, bold, italic symbol; e.g., a, β . Hence the 1 4 and 3 1 matrices,
4321 aaaaa , and
3
2
1
β ,
are both vectors. A is a row vector; β is a column vector.
The numbers (or symbols) inside a matrix are called elements. Thus, in the first matrix, A, given above, a11 and a23 are elements of the matrix A. Similarly, in the second matrix, A, given earlier, the numbers 4 and 6 are elements. Elements in a matrix are indexed (or referred to) by subscripts that give their row and column locations. For, instance, in the first matrix, A, above, a23 is the element in the second row of the third column. Similarly, in the matrix, D, above, element d41 is found in the fourth row, first column. In general, aij is the i,j’th element of A. When referring to elements of vectors, however, the first row (or column) is assumed. Hence, in the vector, a, above, a2 is its second element; in
β , 3 is the third element. In general, elements of vectors, are indicated by the
j’th element of row (i.e., 1 x q) vectors and the i’th element of column (i.e., p x 1) vectors.
A matrix in which the number of rows (p) equals the number of columns (q) is called a square matrix.; otherwise, providing it is not a vector, it is called a rectangular matrix.
The natural order of a rectangular matrix has more rows than columns (i.e., p > q). Hence,
23
22
21
13
12
11
a
a
a
a
a
a
A ,
a 3 x 2 matrix, is presented in its natural order.
The transpose of a matrix is obtained by interchanging its rows and columns. Thus, the transpose of A, represented as A , is
23
13
22
12
21
11
a
a
a
a
a
aA ,
a 3 x 2 matrix. As a concrete example, consider
622
761
475
342
X ,
which, when transposed, becomes
6
2
2
7
6
1
4
7
5
3
4
2
X .
The natural order of a vector is a p x 1 column vector. Here, the 4 x 1 column vector,
4
3
2
1
a
a
a
a
a ,
is presented in its natural order. The transpose of a,
4321 aaaaa ,
is a 1 x 4 row vector.
As mentioned earlier a matrix where p = q is a square matrix. At least two particular types of square matrices are particularly important, symmetric matrices and diagonal matrices. A symmetric matrix is a square matrix in which the elements above the main diagonal are mirror images of the elements below the main diagonal (the main diagonal is that set of elements running from the upper left-hand side of a square matrix to the lower right-hand. In the symmetric matrix V, given below, the elements in bold represent the main diagonal. Note that the elements above the main diagonal are mirror images of the elements below the main diagonal.
59
64
56
6138
6152
3852
V .
Diagonal matrices are square matrices in which all elements except the main diagonal are zero (0). For instance,
61000
04900
00520
00037
D
is a symmetric diagonal matrix.
An important diagonal matrix is the identity matrix. The identity matrix is a diagonal matrix in which all the elements along the main diagonal are unity (1). For example,
100
010
001
I
is a 3 x 3 identity matrix.
Symmetric matrices have important properties. For instance, a symmetric matrix is equal to its transpose. That is, for the matrix given a little earlier, V = V .
Matrix Operations
Addition
The addition of two matrices, A and B, is accomplished by adding corresponding
elements, e.g., aij + bij.; hence, for the 4 x 3 matrices, andA B .
434241
333231
232221
131211
434342424141
333332323131
232322222121
131312121111
434241
333231
232221
131211
434241
333231
232221
131211
ccc
ccc
ccc
ccc
bababa
bababa
bababa
bababa
bbb
bbb
bbb
bbb
aaa
aaa
aaa
aaa
BAC
It is obvious (or should be obvious) that the orders of the two matrices being added need to be identical. Here, for instance, both matrices are of order 4 x 3. When the orders of the two matrices are not identical, addition is not impossible.
Addition of matrices is commutative. Hence A + B = B + A.
As a concrete example, let
431
854
113
857
A , and
313
214
331
302
B .
Then, C = A + B = B + A
=
341331
281544
313113
380527
=
744
1068
444
1159
.
Multiplication
Multiplication is somewhat more complicated. When multiplying matrices we need to distinguish among multiplication by a scalar, vector multiplication, pre- multiplication, and post-multiplication, inner-products and outer-products. But first, let us define a scalar. A scalar is simply a single number, such as 2, 17.6, or -300.335. Symbolically, we typically use the symbol, c, to represent a scalar.
Scalar multiplication
Scalar multiplication is easily shown by example. Given the 3 x 2 matrix X and the scalar, c, the product, cX, is given by
Note that all the elements in X are multiplied by the scalar, c.
.
3231
22
3231
22
3231
22
3231
22
cxcx
cxc
cc
c
xx
x
cxcx
cx
xx
xc
21
1211
21
1211
21
1211
21
1211
x
xx
x
xx
cx
cxcx
x
xx
cX
As a concrete example, let
21
03
74
X
and c = 5. Then
105
015
3520
21
03
74
21
03
74
5
5Xc
In the above expression, the product, cX, results from pre-multiplying X by the scalar c; whereas the product Xc results from post-multiplying X by the scalar, c. Since scalar multiplication is commutative, cX will always be equal to Xc. This is not the case with matrix multiplication, however. Only in certain special cases will the products formed by pre-multiplying and post-multiplying two matrices, X and
Y be equal. In general XY ≠ YX.
Matrix Multiplication
First, note that given two matrices, e.g., A of order n x m, and B of order p x q, multiplication is only possible when the number of columns in the pre-multiplier is equal to the number of rows in the post-multiplier. Hence the product, AB, is only possible when m = p. Similarly, the product BA is only possible when n = q.
When m = p the product, C = AB will have order n x q. Similarly, when q = n, the product, C = BA will have order p x m.
We can illustrate this by defining the two matrices,
,
434241
333231
232221
131211
xxx
xxx
xxx
xxx
X and
23
13
22
12
21
11
y
y
y
y
y
yY .
Note that X has order 4 x 3, and Y’ has order 2 x 3. Here, only the products, XY (not XY’) and Y’X’ (not YX) are possible. Post-multiplying X by Y, yields
234322422141134312421141
233322322131133312321131
232322222121132312221121
231322122111131312121111
2313
2212
2111
434241
333231
232221
131211
yxyxyxyxyxyx
yxyxyxyxyxyx
yxyxyxyxyxyx
yxyxyxyxyxyx
yy
yy
yy
xxx
xxx
xxx
xxx
XY
where the product is of order 4 x 2. As a more concrete example, let
231
012
131
102
X and
2
1
0
1
3
2Y .
Then
,
77
65
56
95
403232
006014
203132
306104
21
01
32
231
012
131
102
XY
a 4 x 2 matrix. The only other product possible between these two matrices is Y’X’:
,7658
7563
403
232
006
014
203
132
206
102
2
3
1
0
1
2
1
3
1
1
0
2
432342224121332343223121232322222121132312222121
431342124111331343123111231322122111131312121111
43332313
42322212
41312111
23
13
22
12
21
11
2
1
0
1
3
2
xyxyxyxyxyxyxyxyxyxyxyxy
xyxyxyxyxyxyxyxyxyxyxyxy
xxxx
xxxx
xxxx
y
y
y
y
y
yXY
which is a 2 x 4 matrix.
When A and B are both square matrices of the same order, then all the products, AB, BA, A’B, AB’, B’A, BA’, A’B’, and B’A’ are possible. Furthermore, except is certain special cases, the matrices formed by these various products will be different from each other. As an exercise, try computing all possible products of the following two square matrices:
120
132
321
A and
112
013
122
B .
Vector Multiplication
Having learned how to compute matrix multiplication, vector multiplication is easy. It follows the same rules of matrix multiplication except that here one of the matrices is a vector. For instance, given the 4 x 1 column vector,
4
3
2
1
n
n
n
n
n , and the 4 x 2 matrix,
4241
3231
2221
1211
yy
yy
yy
yy
Y ,
the products n Y and Y n are the only possible products. Hence,
.21
424323222121414313212111
4241
3231
2221
1211
4321
iiii ynyn
ynynynynynynynyn
yy
yy
yy
yy
nnnnYn
To put this in concrete terms, let
5213n and .
4
2
3
1
4
2
2
3
Y
Then
25321543320426
3
2
3
1
4
2
2
3
5213
Yn .
Some special cases of vector and matrix multiplication are particularly important. For instance, let the vector, 1 be defined as a vector with all elements equal to 1:
1
1
1
1
1
1 and .
24
13
04
21
32
21
yyY {Note, y1 and y2 are column vectors}.
Then,
81421 yyY1 .
If we let
21
21
31
11
21
21 xxX and
34
13
04
21
32
21 yyY .
Then,
.1631
914
21
21
xyxy
yyYX
Note also that
.2210
1052
2
2
22
212
211
x
x
x
n
xxx
xxxXX
Inner-products and outer-products
Given the two matrices,
3231
2221
1211
aa
aa
aa
A and
232221
131211
bbb
bbbB ,
Verify that the following two products are possible, AB and BA (of course, A’B’ and B’A’ also are possible). The orders of the two products are not identical. AB has order, 3 x 3, while BA has order 2 x 2. The product AB is an outer-product while the product BA is an inner-product. In general, whenever both an inner-product and an outer-product exist for two matrices, the order of the inner-product will be less than the outer-product. Furthermore, in general, the inner-product will not be identical to the outer-product.
For example, let
13
23
12
A and
123
421B .
Then
1386
14109
965
AB and
815
920BA .
Inner- and outer-products are particularly important in vector multiplication. Let
1111a , ,111b and
112
234
432
321
X .
Then
3211099 iii xxxXa , {Note the summation is over
rows} and
j
j
j
j
x
x
x
x
4
3
2
1
4
9
9
6
Xb . {Here the summation is over columns}.
Also, it is easy to verify that
ijx28Xba where the summation is over rows and columns.