Textbook: “Matrix algebra useful for statistics”, Searle



Textbook: “Matrix algebra useful for statistics”, Searle.

Webpage: Webpage:

1. course notes:

,

then click on 統計教材 and then click on

and then click on

Math Algebra ( Word , PDF )

2. Online grades:



Then, click on

• Online Grade: 2008, Summer, Basic Statistics

Objective: introduce basic concepts and skills in matrix algebra. In addition, some applications of matrix algebra in statistics are described.

Section 1. Introduction and Matrix Operations

Definition of [pic] matrix:

An [pic] matrix A is a rectangular array of rc real numbers arranged in r horizontal rows and c vertical columns:

[pic].

The i’th row of A is

[pic],

and the j’th column of A is

[pic]

We often write A as

[pic].

Matrix addition:

Let

[pic],

[pic],

[pic].

Then,

[pic],

[pic]

and the transpose of A is denoted as

[pic]

Example 1:

Let

[pic] and [pic].

Then,

[pic],

[pic]

and

[pic].

Matrix multiplication:

We first define the dot product or inner product of n-vectors.

Definition of dot product:

The dot product or inner product of the n-vectors

[pic] and [pic],

are

[pic].

Example 1:

Let [pic] and [pic]. Then, [pic].

Definition of matrix multiplication:

[pic]

[pic]

[pic]

That is,

[pic]

Example 2:

[pic].

Then,

[pic]

since

[pic], [pic]

[pic], [pic]

[pic], [pic].

Example 3

[pic]

Another expression of matrix multiplication:

[pic]

where [pic] are [pic] matrices.

Example 2 (continue):

[pic]

Note:

Heuristically, the matrices A and B, [pic] and [pic], can be thought as [pic] and [pic] vectors. Thus,

[pic]

can be thought as the multiplication of [pic] and [pic] vectors. Similarly,

[pic]

can be thought as the multiplication of [pic] and [pic] vectors.

Note:

I. [pic] is not necessarily equal to [pic]. For instance, [pic]

[pic].

II. [pic] might be not equal to [pic]. For instance,

[pic]

[pic]

III. [pic], it is not necessary that [pic] or [pic]. For instance,

[pic]

[pic]

IV. [pic], [pic], [pic]

p factors

Also, [pic] is not necessarily equal to [pic].

V. [pic].

Trace:

Definition of the trace of a matrix:

The sum of the diagonal elements of a [pic] square matrix is called the trace of the matrix, written [pic], i.e., for

[pic],

[pic].

Example 4:

Let [pic]. Then, [pic].

Homework 1

1. Prove [pic], where A and B are [pic] and [pic]

matrices, respectively.

2.

(a) When does [pic]

(b) When [pic] Prove [pic]

(c) When [pic], prove [pic]

Section 2 Special Matrices

2.1 Symmetric Matrices:

Definition of symmetric matrix:

A [pic] matrix [pic] is defined as symmetric if [pic]. That is,

[pic].

Example 1:

[pic] is symmetric since [pic].

Example 2:

Let [pic] be random variables. Then,

[pic] [pic] … [pic]

[pic]

is called the covariance matrix, where [pic], is the covariance of the random variables [pic] and [pic] and [pic] is the variance of [pic]. V is a symmetric matrix. The correlation matrix for [pic] is defined as

[pic] [pic] … [pic]

[pic]

where [pic], is the correlation of [pic] and [pic]. R is also a symmetric matrix. For instance, let [pic] be the random variable represent the sale amount of some product and [pic] be the random variable represent the cost spent on advertisement. Suppose

[pic]

Then,

[pic]

and

[pic]

Example 3:

Let [pic] be a [pic] matrix. Then, both [pic] and [pic] are symmetric since

[pic] and [pic].

[pic] is a [pic] symmetric matrix while [pic] is a [pic] symmetric matrix.

[pic]

Also,

[pic]

Similarly,

[pic]

and

[pic]

For instance, let

[pic] and [pic].

Then,

[pic]

In addition,

[pic]

Note:

A and B are symmetric matrices. Then, AB is not necessarily equal to [pic]. That is, AB might not be a symmetric matrix.

Example 4:

[pic] and [pic].

Then,

[pic]

Properties of [pic] and [pic]:

(a)

[pic]

(b)

[pic]

[proof]

(a)

Let

[pic]

[pic].

Thus, for [pic]

[pic]

[pic]

[pic]

[pic]

[pic]

[pic]

(b)

Since [pic]

[pic]

By (a),

[pic]

Note:

A [pic] matrix [pic] is defined as skew-symmetric if [pic]. That is,

[pic].

Example 5:

[pic]

Thus,

[pic][pic].

2.2 Idempotent Matrices:

Definition of idempotent matrices:

A square matrix K is said to be idempotent if

[pic]

Properties of idempotent matrices:

1. [pic] for r being a positive integer.

2. [pic] is idempotent.

3. If [pic] and [pic] are idempotent matrices and [pic]. Then,

[pic] is idempotent.

[proof:]

1.

For [pic] [pic].

Suppose [pic]is true, then [pic].

By induction, [pic] for r being any positive integer.

2.

[pic]

3.

[pic]

Example 1

Let [pic] be a [pic] matrix. Then,

[pic] is an idempotent matrix since

[pic].

Note:

A matrix A satisfying [pic] is called nilpotent, and that for which [pic] could be called unipotent.

Example 2:

[pic] [pic] A is nilpotent.

[pic] [pic] B is unipotent.

Note:

[pic] is a idempotent matrix. Then, [pic] might not be idempotent.

2.3 Orthogonal Matrices:

Definition of orthogonality:

Two [pic] vectors u and v are said to be orthogonal if

[pic]

A set of [pic] vectors [pic] is said to be orthonormal if

[pic]

Definition of orthogonal matrix:

A [pic] square matrix P is said to be orthogonal if

[pic].

Note:

[pic]

[pic] [pic]

Thus,

[pic] and [pic]

are both orthonormal sets!!

Example 1:

(a) Helmert Matrices:

The Helmert matrix of order n has the first row

[pic],

and the other n-1 rows ([pic]) has the form,

[pic]

(i-1) items n-i items

For example, as [pic], then

[pic]

In statistics, we can use H to find a set of uncorrelated random variables. Suppose [pic] are random variables with

[pic]

Let

[pic]

Then,

[pic]

since [pic] is an orthonormal set of vectors. That is, [pic] are uncorrelated random variables. Also,

[pic],

where

[pic].

(b) Givens Matrices:

Let the orthogonal matrix be

[pic]

G is referred to as a Givens matrix of order 2. For a Givens matrix of order 3, there are [pic] different forms,

1 2 3 1 2 3

[pic].

The general form of a Givens matrix [pic] of order 3 is an identity matrix except for 4 elements, [pic] and [pic] are in the i’th and j’th rows and columns. Similarly, For a Givens matrix of order 4, there are [pic] different forms,

1 2 3 4 1 2 3 4

[pic]

1 2 3 4 1 2 3 4

[pic]

1 2 3 4 1 2 3 4

[pic].

For the Givens matrix of order n, here are [pic] different forms. The general form of [pic] is an identity matrix except for 4 elements,

[pic].

2.4 Positive Definite Matrices:

Definition of positive definite matrix:

A symmetric [pic] matrix A satisfying

[pic] for all [pic],

is referred to as a positive definite (p.d.) matrix.

Intuition:

If [pic] for all real numbers x, [pic], then the real number a is positive. Similarly, as x is a [pic] vector, A is a [pic] matrix and [pic], then the matrix A is “positive”.

Note:

A symmetric [pic] matrix A satisfying

[pic] for all [pic],

is referred to as a positive semidefinite (p.d.) matrix.

Example 1:

Let

[pic] and [pic].

Thus,

[pic]

Let [pic]. Then, A is positive semidefinite since for [pic]

[pic].

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download