1.10.5 Covariance and Correlation - Queen Mary University of London

1.10. TWO-DIMENSIONAL RANDOM VARIABLES

41

1.10.5 Covariance and Correlation

Covariance and correlation are two measures of the strength of a relationship between two r.vs.

We will use the following notation. E(X1) = ?X1 E(X2) = ?X2 var(X1) = X2 1 var(X2) = X2 2

Also, we assume that X2 1 and X2 2 are finite positive values. A simplified notation ?1, ?2, 12, 22 will be used when it is clear which rvs we refer to.

Definition 1.19. The covariance of X1 and X2 is defined by cov(X1, X2) = E[(X1 - ?X1)(X2 - ?X2)].

(1.16)

Some useful properties of the covariance and correlation are given in the following two theorems.

Theorem 1.15. Let X1 and X2 denote random variables and let a, b, c, . . . denote some constants. Then, the following properties hold.

1. cov(X1, X2) = E(X1X2) - ?X1?X2. 2. If random variables X1 and X2 are independent then

cov(X1, X2) = 0.

3. var(aX1 + bX2) = a2 var(X1) + b2 var(X2) + 2ab cov(X1, X2). 4. For U = aX1 + bX2 + e and for V = cX1 + dX2 + f we can write

cov(U, V ) = ac var(X1) + bd var(X2) + (ad + bc) cov(X1, X2).

42 CHAPTER 1. ELEMENTS OF PROBABILITY DISTRIBUTION THEORY

Proof. We will show property 4.

cov(U, V ) = E (aX1 + bX2 + e) - E(aX1 + bX2 + e) ? (cX1 + dX2 + f ) - E(cX1 + dX2 + f )

= E a(X1 - E X1) + b(X2 - E X2) c(X1 - E X1) + d(X2 - E X2) = E ac(X1 - E X1)2 + bd(X2 - E X2)2 + (ad + bc)(X1 - E X1)(X2 - E X2) = ac var X1 + bd var X2 + (ad + bc) cov(X1, X2).

Property 2 says that if two variables are independent, then their covariance is zero. This does not always work both ways, that is it does not mean that if the covariance is zero then the variables must be independent. The following small example shows this fact.

Example 1.27. Let X U(-1, 1) and let Y = X2. Then

E(X) = 0

E(Y ) = E(X2) = 1 x2 1 dx = 1

-1 2

3

E(XY ) = E(X3) = 0.

Hence,

cov(X, Y ) = E(XY ) - E(X) E(Y ) = 0,

but Y is a function of X, so these two variables are not independent.

Another measure of strength of relationship of two rvs is correlation. It is defined as

Definition 1.20. The correlation of X1 and X2 is defined by

(X1,X2)

=

cov(X1, X2) . X1 X2

(1.17)

Correlation is a dimensionless measure and it expresses the strength of linearly related variables as shown in the following theorem.

1.10. TWO-DIMENSIONAL RANDOM VARIABLES

43

Theorem 1.16. For any random variables X1 and X2

1. -1 (X1,X2) 1, 2. |(X1,X2)| = 1 iff there exist numbers a = 0 and b such that

P (X2 = aX1 + b) = 1. If (X1,X2) = 1 then a > 0, and if (X1,X2) = -1 then a < 0.

Exercise 1.17. Prove Theorem 1.16. Hint: Consider roots (and so the discriminant) of the quadratic function of t:

g(t) = E[(X - ?X)t + (Y - ?Y )]2.

Example 1.28. Let the joint pdf of X, Y be

fX,Y (x, y) = 1 on the support {(x, y) : 0 < x < 1, x < y < x + 1}.

The two rvs are not independent as the range of Y depends on x. We will calculate the correlation between X and Y . For this we need to obtain the marginal pdfs, fX(x) and fY (y). The marginal pdf for X is

x+1

fX (x) =

1dy = y|xx+1 = 1, on support {x : 0 < x < 1}.

x

Note that to obtain the marginal pdf for the rv Y the range of X has to be considered separately for y (0, 1) and for y [1, 2). When y (0, 1) then x (0, y). When y [1, 2), then x (y - 1, 1). Hence,

y 0

1dx

=

x|y0

=

y,

fY (y) =

1 y-1

1dx

=

x|1y-1

=

2

-

y,

0,

for y (0, 1); for y [1, 2); otherwise.

These give

?X

=

1, 2

?Y = 1

X2

=

(b - a)2 12

=

1 ;

12

Y2

= E(Y 2) - [E(Y )]2

=

7 6

-1=

1, 6

44 CHAPTER 1. ELEMENTS OF PROBABILITY DISTRIBUTION THEORY

where Also, Hence,

E(Y 2) =

1

y2ydy +

2 y2(2 - y)dy = 7 .

0

1

6

E(XY ) =

1

x+1

xy1dydx =

7.

0x

12

cov(X, Y ) = E(XY ) - E(X) E(Y ) = 7 - 1 ? 1 = 1 .

12 2

12

Finally,

(X,Y )

=

cov(X, Y X Y

)

=

1 12

= 1 .

11

2

12 6

The linear relationship between X and Y is not very strong.

Note: We can make an interesting comparison of this value of the correlation with the correlation of X and Y having a joint uniform distribution on {(x, y) : 0 < x < 1, x < y < x + 0.1}, which is a 'narrower strip' of values then previously. Then, fX,Y (x, y) = 10 and it can be shown, that (X, Y ) = 10/ 101, which is close to 1. The linear relationship between X and Y is very strong in this case.

1.10.6 Bivariate Normal Distribution

Here we use matrix notation. A bivariate rv is treated as a random vector

X=

X1 X2

.

The expectation of a bivariate random vector is written as

? = EX = E

X1 X2

=

?1 ?2

and its variance-covariance matrix is

V=

var(X1) cov(X1, X2) cov(X2, X1) var(X2)

=

12 12 12 22

.

1.10. TWO-DIMENSIONAL RANDOM VARIABLES

45

Figure 1.2: Bivariate Normal pdf

Then the joint pdf of a normal bi-variate rv X is given by

fX(x) = 2

1 exp

det(V )

- 1 (x - ?)TV -1(x - ?) 2

,

where x = (x1, x2)T.

(1.18)

The determinant of V is det V = det

12 12 12 22

= (1 - 2)1222.

Hence, the inverse of V is

V -1 = 1 det V

22

-12

-12 12

1 = 1 - 2

1-2

-1-12-1

-1-12-1

2-2

.

Then the exponent in formula (1.18) can be written as

-

1 (x

-

?)TV

-1(x

-

?)

=

2

=

- 2(1

1 -

2) (x1

-

?1,

x2

-

?2)

1-2

-1-12-1

-1-12-1

2-2

x1 - ? x2 - ?

=

- 2(1

1 -

2)

(x1

- ?1)2 12

-

2 (x1

-

?1)(x2 12

-

?2)

+

(x2

- ?2)2 22

.

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download