Matrix Notation and Operations

[Pages:16]D.G. Bonett (6/2018)

Matrix Notation and Operations

Matrix Notation

An r ? c matrix is a rectangular array of elements with r rows and c columns. An r ? c matrix is said to be of order r ? c. A matrix is usually denoted by a capital letter printed in a boldface font (e.g., A, B, X). The elements of the matrix are represented by lower case letters with a double subscript (e.g., , , ). For instance, in the matrix X, 13 is the element in the first row and the third column. A 4 ? 3 matrix X is shown below.

11 12 13

X = [2311

22 32

23 33

]

41 42 43

A matrix with a single row is called a row vector and a matrix with a single column is called a column vector. Vectors are usually represented by lower case letters printed in a boldface font (e.g., a, b, x). The elements of the vector are represented by lower case letters with a single subscript (e.g., , , ). A 3 ? 1 column vector y and a 1 ? 4 row vector h are shown below.

1 y = [2]

3

h = [1 2 3 4 ].

It is sometimes necessary to refer to a particular row or column of a matrix. These row or column vectors are represented by a subscripted lower case letter in a boldface font. For instance, the jth row vector in the above 4 ? 3 matrix X would be noted as .

A square matrix has the same number of rows as columns. A square matrix where the diagonal elements (the elements where the two subscripts are equal) are nonzero and the off-diagonal elements are zero is called a diagonal matrix. A 3 ? 3 diagonal matrix D is shown below.

1

D.G. Bonett (6/2018)

11 0 0 D = [ 0 22 0 ]

0 0 33

The identity matrix is a special type of diagonal matrix where all diagonal elements are equal to 1. The identity matrix is usually represented as I or In where n is the order of the identity matrix.

A square matrix where the jkth element is equal to the kjth element is called a symmetric matrix. A symmetric 3 ? 3 matrix is shown below.

14 5 2 S = [ 5 20 8 ]

2 8 11

A one vector is a row or column vector in which every element is equal to 1 and is represented as the number one printed in a boldface font. A 1 ? 3 one vector is shown below.

1 = [1 1 1]

Matrix Operations

The transpose of a matrix X is represented as X' (or XT). The transpose of a matrix is obtained by interchanging the rows and columns of the matrix. For instance, if

4 6 X = [7 1]

3 9 then

X' = [46

7 1

39] .

Note that the jkth element in X is equal to the kjth element in X'. Most vectors in statistical formulas are assumed to be column vectors. Row vectors, when needed, are obtained by taking the transpose of a column vector.

2

D.G. Bonett (6/2018)

If two matrices A and B are of the same order, the two matrices are then conformable for addition or subtraction, and A + B is a matrix with element + in the jth row and the kth column, as illustrated below for the sum of two 3 ? 2 matrices.

A + B = [1211

12 22

13 23

]

+

[1211

12 22

1233]

=

[2111

+ +

11 11

12 + 12 22 + 22

13 23

+ +

1233]

Likewise, A ? B is a matrix with element ? in the jth row and the kth column, as illustrated below for the difference of two 3 ? 2 matrices.

A ? B = [1211

12 22

1233] ? [1211

12 22

1233]

=

[2111

- -

11 11

12 - 12 22 - 22

13 23

- -

13 23

]

To multiply a matrix by a scalar (i.e., a single number), simply multiply each element in the matrix by the scalar. Scalars are represented by italicized lower case letters in a nonboldface font. To illustrate, if b = 2 and

A = [46

7 1

39]

then

bA

=

[

8 12

14 2

168].

Some statistical formulas involve the subtraction of a scalar from an n x 1 vector. The result is obtained by first multiplying the scalar by an n x 1 one vector and then taking the difference of the two vectors. For instance, if y is a 3 ? 1 vector and m is a scalar, then y ? m is

1

1 1 -

y ? m1 = [2] - [1] = [2 - ]

3

1 3 -

3

D.G. Bonett (6/2018)

The dot product of a n ? 1 vector a with a n ? 1 vector b is

a'b = 11 + 22 + ... +

Note that a'b = b'a. For instance, if a' = [4 3 2] and b' = [6 1 4], then a'b = 4(6) + 3(1) + 2(4) = 35.

Two n ? 1 vectors, a and b, are said to be orthogonal if a'b = 0. For instance, if a' = [.5 .5 -1] and b' = [1 -1 0], then a and b are orthogonal because 'b = (.5)(1) + (.5)(-1) + (-1)(0) = 0.

Two matrices A and B can be multiplied if they are conformable for multiplication. To compute the matrix product AB, the number of columns of A must equal the number of rows of B. In general, if A is r ? n and B is n ? c, then the matrix product AB is an r ? c matrix. The jkth element in the r ? c product matrix is equal to the dot product where is the jth row vector of matrix A and is the kth column vector of matrix B. For instance, the matrices A and B shown below are conformable for computing the product AB because A is 2 ? 3 and B is 3 ? 4 so that the product will be a 2 ? 4 matrix.

A = [46

7 1

93]

1214 B = [5 4 3 1]

4232

Each of the 2 ? 4 = 8 elements of the AB matrix is a dot product. For instance, the element in row 1 and column 1 of the product AB is

1 [4 7 3] [5] = 4(1) + 7(5) + 3(4) = 51

4

and the element in row 2 and column 3 of AB is

1 [6 1 9] [3] = 6(1) + 1(3) + 9(3) = 36.

3

After computing all 8 dot products, the following result is obtained.

AB

=

[

51 47

42 34

34 36

2493]

4

D.G. Bonett (6/2018)

Unlike scalar multiplication where ab = ba, the matrix product AB does not in general equal BA. Regarding the matrix product AB, we can say that B is pre-multiplied by A or that A is post-multiplied by B. The product of matrix A with itself is denoted as A2.

The transpose of a matrix product is equal to the product of the transposed matrices in reverse order. Specifically, (AB)' = B'A'.

The product of three matrices ABC requires A and B to be conformable for multiplication and also requires B and C to be conformable for multiplication. The product ABC can be obtained by first computing AB and then post-multiplying the result by C, or by first computing BC and then pre-multiplying the result by A.

If A is a square matrix, then the matrix inverse of A is represented as A-1. If the inverse of A exists, then AA-1 = I. This result is a generalization of scalar arithmetic where x(1/x) = 1, assuming x 0 so that the inverse of x exists. Computing a matrix inverse is tedious, and the amount of computational effort increases as the size of the matrix increases, but inverting a 2 ? 2 matrix is not difficult. The inverse of a 2 ? 2 matrix A is

A-1

=

(1/d)[

22 -21

-1112 ]

where d = 1122 - 1221 is called the determinant of A. The matrix inverse does not exist unless the determinant is nonzero.

Inverting a diagonal matrix D of any order is simple. The inverse of D is equal to a diagonal matrix where the jth diagonal element is equal to 1/ .

The trace of a square n ? n matrix A, denoted as tr(A), is defined as the sum of its n diagonal elements.

tr(A) = 11 + 22 + ... +

5

D.G. Bonett (6/2018)

For instance if

14 5 2 V = [ 9 20 8 ]

7 8 11

then tr(V) = 14 + 20 + 11 = 45.

The Kronecker product of two matrices, an m ? n matrix A and a p ? q matrix B, is defined to be the mp ? nq matrix

11 12 1

A

B

=

[

21

22

...

2

]

1 2

which is obtained by replacing each element with the p ? q matrix B. For example, if

A

=

[

1 3

-21]

and

b = [1

2

3]

then

A b = [13

2 6

3 9

2 -1

4 -2

-63].

The Kronecker product of an identity matrix and another matrix has the following simple form

I B = [

...

]

where is a matrix of zeros that has the same order as B.

The transpose of a Kronecker product of two matrices is (A B)' = A' B'

6

D.G. Bonett (6/2018)

and the inverse of a Kronecker product of two matrices is (A B)-1 = A-1 B-1.

The product of A B and C D is equal to (A B)(C D) = AC BD

assuming A and C are conformable for multiplication and B and D are conformable for multiplication.

In some statistical formulas, it is convenient to rearrange the elements of an r ? c matrix A into an rc ? 1 column vector a. This is done by stacking the c column vectors (which are each r ? 1) of A (1, 2, ... , ) one under the other as shown below

1 a = [2]

The conversion of a matrix A into a vector is denoted as vec(A). To illustrate, consider the following 4 ? 2 matrix Y and vec(Y). The mat(y) function returns the vector into its original matrix form.

14 2

Y

=

[

9 7

181]

16 34

14

9

7

vec(Y) =

16 2

= y

8

11

[ 34 ]

The vectorization of a matrix triple product can be expressed as

vec(ABC) = (C A)vec(B)

7

D.G. Bonett (6/2018)

and the vectorization of a matrix product AB follows from this result by setting C = I

vec(AB) = (Ip A)vec(B)

where p is the number of columns of B.

For two matrices A and B of the same order, matrix equality A = B indicates that = for every value of j and k. Matrix inequality A B indicates that there is at least one element in A that is not equal to its corresponding element in B.

Covariance Matrices

A covariance matrix is a symmetric matrix with variances in the diagonal elements and covariances in the off-diagonal elements. If r response variables y = [1 2 ... ] are measured for each person in a random sample of n people, the estimated variance for the jth response variable is

2 = =1( - )2/( - 1)

and the estimated covariance between the jth and kth response variables is = =1( - )( - )/( - 1).

The estimated covariance between the jth and kth measurements is also equal to where is the estimated Pearson correlation between the two response variables. Note that = .

The r variances and the r(r ? 1)/2 covariances of y = [1 2 ... ] can be summarized in an r ? r covariance matrix denoted as cov(y). For instance, with r = 3 the covariance matrix is

12 12 13 cov(y) = [12 22 23] .

13 23 32

If there are n sets of response variables where the ith set has a covariance matrix Si, and response variables from different sets are assumed to be uncorrelated, the covariance matrix for all response variables has the following form

8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download