1.10.5 Covariance and Correlation - Queen Mary University of London
1.10. TWO-DIMENSIONAL RANDOM VARIABLES
41
1.10.5 Covariance and Correlation
Covariance and correlation are two measures of the strength of a relationship between two r.vs.
We will use the following notation. E(X1) = ?X1 E(X2) = ?X2 var(X1) = X2 1 var(X2) = X2 2
Also, we assume that X2 1 and X2 2 are finite positive values. A simplified notation ?1, ?2, 12, 22 will be used when it is clear which rvs we refer to.
Definition 1.19. The covariance of X1 and X2 is defined by cov(X1, X2) = E[(X1 - ?X1)(X2 - ?X2)].
(1.16)
Some useful properties of the covariance and correlation are given in the following two theorems.
Theorem 1.15. Let X1 and X2 denote random variables and let a, b, c, . . . denote some constants. Then, the following properties hold.
1. cov(X1, X2) = E(X1X2) - ?X1?X2. 2. If random variables X1 and X2 are independent then
cov(X1, X2) = 0.
3. var(aX1 + bX2) = a2 var(X1) + b2 var(X2) + 2ab cov(X1, X2). 4. For U = aX1 + bX2 + e and for V = cX1 + dX2 + f we can write
cov(U, V ) = ac var(X1) + bd var(X2) + (ad + bc) cov(X1, X2).
42 CHAPTER 1. ELEMENTS OF PROBABILITY DISTRIBUTION THEORY
Proof. We will show property 4.
cov(U, V ) = E (aX1 + bX2 + e) - E(aX1 + bX2 + e) ? (cX1 + dX2 + f ) - E(cX1 + dX2 + f )
= E a(X1 - E X1) + b(X2 - E X2) c(X1 - E X1) + d(X2 - E X2) = E ac(X1 - E X1)2 + bd(X2 - E X2)2 + (ad + bc)(X1 - E X1)(X2 - E X2) = ac var X1 + bd var X2 + (ad + bc) cov(X1, X2).
Property 2 says that if two variables are independent, then their covariance is zero. This does not always work both ways, that is it does not mean that if the covariance is zero then the variables must be independent. The following small example shows this fact.
Example 1.27. Let X U(-1, 1) and let Y = X2. Then
E(X) = 0
E(Y ) = E(X2) = 1 x2 1 dx = 1
-1 2
3
E(XY ) = E(X3) = 0.
Hence,
cov(X, Y ) = E(XY ) - E(X) E(Y ) = 0,
but Y is a function of X, so these two variables are not independent.
Another measure of strength of relationship of two rvs is correlation. It is defined as
Definition 1.20. The correlation of X1 and X2 is defined by
(X1,X2)
=
cov(X1, X2) . X1 X2
(1.17)
Correlation is a dimensionless measure and it expresses the strength of linearly related variables as shown in the following theorem.
1.10. TWO-DIMENSIONAL RANDOM VARIABLES
43
Theorem 1.16. For any random variables X1 and X2
1. -1 (X1,X2) 1, 2. |(X1,X2)| = 1 iff there exist numbers a = 0 and b such that
P (X2 = aX1 + b) = 1. If (X1,X2) = 1 then a > 0, and if (X1,X2) = -1 then a < 0.
Exercise 1.17. Prove Theorem 1.16. Hint: Consider roots (and so the discriminant) of the quadratic function of t:
g(t) = E[(X - ?X)t + (Y - ?Y )]2.
Example 1.28. Let the joint pdf of X, Y be
fX,Y (x, y) = 1 on the support {(x, y) : 0 < x < 1, x < y < x + 1}.
The two rvs are not independent as the range of Y depends on x. We will calculate the correlation between X and Y . For this we need to obtain the marginal pdfs, fX(x) and fY (y). The marginal pdf for X is
x+1
fX (x) =
1dy = y|xx+1 = 1, on support {x : 0 < x < 1}.
x
Note that to obtain the marginal pdf for the rv Y the range of X has to be considered separately for y (0, 1) and for y [1, 2). When y (0, 1) then x (0, y). When y [1, 2), then x (y - 1, 1). Hence,
y 0
1dx
=
x|y0
=
y,
fY (y) =
1 y-1
1dx
=
x|1y-1
=
2
-
y,
0,
for y (0, 1); for y [1, 2); otherwise.
These give
?X
=
1, 2
?Y = 1
X2
=
(b - a)2 12
=
1 ;
12
Y2
= E(Y 2) - [E(Y )]2
=
7 6
-1=
1, 6
44 CHAPTER 1. ELEMENTS OF PROBABILITY DISTRIBUTION THEORY
where Also, Hence,
E(Y 2) =
1
y2ydy +
2 y2(2 - y)dy = 7 .
0
1
6
E(XY ) =
1
x+1
xy1dydx =
7.
0x
12
cov(X, Y ) = E(XY ) - E(X) E(Y ) = 7 - 1 ? 1 = 1 .
12 2
12
Finally,
(X,Y )
=
cov(X, Y X Y
)
=
1 12
= 1 .
11
2
12 6
The linear relationship between X and Y is not very strong.
Note: We can make an interesting comparison of this value of the correlation with the correlation of X and Y having a joint uniform distribution on {(x, y) : 0 < x < 1, x < y < x + 0.1}, which is a 'narrower strip' of values then previously. Then, fX,Y (x, y) = 10 and it can be shown, that (X, Y ) = 10/ 101, which is close to 1. The linear relationship between X and Y is very strong in this case.
1.10.6 Bivariate Normal Distribution
Here we use matrix notation. A bivariate rv is treated as a random vector
X=
X1 X2
.
The expectation of a bivariate random vector is written as
? = EX = E
X1 X2
=
?1 ?2
and its variance-covariance matrix is
V=
var(X1) cov(X1, X2) cov(X2, X1) var(X2)
=
12 12 12 22
.
1.10. TWO-DIMENSIONAL RANDOM VARIABLES
45
Figure 1.2: Bivariate Normal pdf
Then the joint pdf of a normal bi-variate rv X is given by
fX(x) = 2
1 exp
det(V )
- 1 (x - ?)TV -1(x - ?) 2
,
where x = (x1, x2)T.
(1.18)
The determinant of V is det V = det
12 12 12 22
= (1 - 2)1222.
Hence, the inverse of V is
V -1 = 1 det V
22
-12
-12 12
1 = 1 - 2
1-2
-1-12-1
-1-12-1
2-2
.
Then the exponent in formula (1.18) can be written as
-
1 (x
-
?)TV
-1(x
-
?)
=
2
=
- 2(1
1 -
2) (x1
-
?1,
x2
-
?2)
1-2
-1-12-1
-1-12-1
2-2
x1 - ? x2 - ?
=
- 2(1
1 -
2)
(x1
- ?1)2 12
-
2 (x1
-
?1)(x2 12
-
?2)
+
(x2
- ?2)2 22
.
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- exercise 1 2 page no 1 byju s
- exercise 4 1 solutions
- rational numbers national council of educational research and training
- math 221 1st semester calculus department of mathematics
- exercise 1 2 solutions calculus and analytic geometry mathematics 12
- exercise 7 2 solutions
- wow maths class 7 book pdf free download
- exercise 2 6 solutions
- chap 2 8th nov national council of educational research and training
- exercise 1