Chapter 3 Random Vectors and Multivariate Normal Distributions

[Pages:29]Chapter 3 Random Vectors and Multivariate Normal Distributions

3.1 Random vectors

Definition 3.1.1. Random vector. Random vectors are vectors of random

83

BIOS 2083

Linear Models

Abdus S. Wahed

variables. For instance,

X

=

X1

X2 ...

,

Xn

where each element represent a random variable, is a random vector.

Definition 3.1.2. Mean and covariance matrix of a random vector.

The mean (expectation) and covariance matrix of a random vector X is de-

fined as follows: and

E

[X]

=

E E

[X1]

[X2] ...

,

E [Xn]

cov(X) = E {X - E (X)} {X - E (X)}T

=

12

21 ...

12

22 ...

...

... ...

1n

2n ...

,

n1 n2 . . . n2

(3.1.1)

where j2 = var(Xj) and jk = cov(Xj, Xk) for j, k = 1, 2, . . . , n.

Chapter 3

84

BIOS 2083

Linear Models

Abdus S. Wahed

Properties of Mean and Covariance.

1. If X and Y are random vectors and A, B, C and D are constant matrices, then

E [AXB + CY + D] = AE [X] B + CE[Y] + D.

(3.1.2)

Proof. Left as an exercise.

2. For any random vector X, the covariance matrix cov(X) is symmetric.

Proof. Left as an exercise.

3. If Xj, j = 1, 2, . . . , n are independent random variables, then cov(X) = diag(j2, j = 1, 2, . . . , n).

Proof. Left as an exercise.

4. cov(X + a) = cov(X) for a constant vector a.

Proof. Left as an exercise.

Chapter 3

85

BIOS 2083

Linear Models

Properties of Mean and Covariance (cont.)

5. cov(AX) = Acov(X)AT for a constant matrix A.

Proof. Left as an exercise.

6. cov(X) is positive semi-definite.

Proof. Left as an exercise. 7. cov(X) = E[XXT ] - E[X] {E[X]}T .

Proof. Left as an exercise.

Abdus S. Wahed

Chapter 3

86

BIOS 2083

Linear Models

Abdus S. Wahed

Definition 3.1.3. Correlation Matrix.

A correlation matrix of a vector of random variable X is defined as the

matrix of pairwise correlations between the elements of X. Explicitly,

corr(X)

=

1

21 ...

12

1 ...

...

... ...

1n

2n ...

,

(3.1.3)

n1 n2 . . . 1

where jk = corr(Xj, Xk) = jk/(jk), j, k = 1, 2, . . . , n.

Example 3.1.1. If only successive random variables in the random vector X

are correlated and have the same correlation , then the correlation matrix

corr(X) is given by

corr(X)

=

1 0 ...

1 ...

0 1 ...

... ... ... ...

0 0 0 ...

,

0 0 0 ... 1

(3.1.4)

Chapter 3

87

BIOS 2083

Linear Models

Abdus S. Wahed

Example 3.1.2. If every pair of random variables in the random vector X

have the same correlation , then the correlation matrix corr(X) is given by

corr(X)

=

1 ...

1 ...

1 ...

... ... ... ...

...

,

... 1

(3.1.5)

and the random variables are said to be exchangeable.

3.2 Multivariate Normal Distribution

Definition 3.2.1. Multivariate Normal Distribution. A random vector X = (X1, X2, . . . , Xn)T is said to follow a multivariate normal distribution with mean and covariance matrix if X can be expressed as

X = AZ + ,

where = AAT and Z = (Z1, Z2, . . . , Zn) with Zi, i = 1, 2, . . . , n iid N (0, 1) variables.

Chapter 3

88

BIOS 2083

Linear Models

Abdus S.Wahed

Bivariate normal distribution with mean (0, 0)T and covariance matrix 0.25 0.3 0.3 1.0

Probability Density

0.4 0.3 0.2 0.1

0 2

0

-2 x2

3

2

1

0

-1

-2

-3

x1

Definition 3.2.2. Multivariate Normal Distribution. A random vector X = (X1, X2, . . . , Xn)T is said to follow a multivariate normal distribution with mean and a positive definite covariance matrix if X has the density

1 fX(x) = (2)n/2||1/2 exp

-1 (x - )T -1 (x - ) 2

(3.2.1)

.

Chapter 3

89

BIOS 2083

Linear Models

Abdus S. Wahed

Properties

1. Moment generating function of a N (, ) random variable X is given

by

MX(t) = exp

T t + 1tT t 2

.

(3.2.2)

2. E(X) = and cov(X) = .

3. If X1, X2, . . . , Xn are i.i.d N (0, 1) random variables, then their joint distribution can be characterized by X = (X1, X2, . . . , Xn)T N (0, In).

4. X Nn(, ) if and only if all non-zero linear combinations of the components of X are normally distributed.

Chapter 3

90

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download