General Bivariate Normal - Duke University

6.5

Conditional Distributions

General Bivariate Normal

Let Z1 , Z2 ¡« N (0, 1), which we will use to build a general bivariate normal

distribution.





1 2

1

2

exp ? (z1 + z2 )

f (z1 , z2 ) =

2¦Ð

2

Lecture 22: Bivariate Normal Distribution

Statistics 104

We want to transform these unit normal distributions to have the follow

arbitrary parameters: ?X , ?Y , ¦ÒX , ¦ÒY , ¦Ñ

Colin Rundel

April 11, 2012

X = ¦ÒX Z1 + ? X

p

Y = ¦ÒY [¦ÑZ1 + 1 ? ¦Ñ2 Z2 ] + ?Y

Statistics 104 (Colin Rundel)

6.5

Lecture 22

Conditional Distributions

6.5

April 11, 2012

1 / 22

Conditional Distributions

General Bivariate Normal - Marginals

General Bivariate Normal - Cov/Corr

First, lets examine the marginal distributions of X and Y ,

Second, we can find Cov (X , Y ) and ¦Ñ(X , Y )

Cov (X , Y ) = E [(X ? E (X ))(Y ? E (Y ))]

h

i

p

= E (¦ÒX Z1 + ?X ? ?X )(¦ÒY [¦ÑZ1 + 1 ? ¦Ñ2 Z2 ] + ?Y ? ?Y )

i

h

p

= E (¦ÒX Z1 )(¦ÒY [¦ÑZ1 + 1 ? ¦Ñ2 Z2 ])

i

h

p

= ¦ÒX ¦ÒY E ¦ÑZ12 + 1 ? ¦Ñ2 Z1 Z2

X = ¦ÒX Z1 + ?X

= ¦ÒX N (0, 1) + ?X

= N (?X , ¦ÒX2 )

p

1 ? ¦Ñ2 Z2 ] + ?Y

p

= ¦ÒY [¦ÑN (0, 1) + 1 ? ¦Ñ2 N (0, 1)] + ?Y

Y = ¦ÒY [¦ÑZ1 +

= ¦ÒX ¦ÒY ¦ÑE [Z12 ]

= ¦ÒX ¦ÒY ¦Ñ

= ¦ÒY [N (0, ¦Ñ2 ) + N (0, 1 ? ¦Ñ2 )] + ?Y

¦Ñ(X , Y ) =

= ¦ÒY N (0, 1) + ?Y

Cov (X , Y )

=¦Ñ

¦ÒX ¦ÒY

= N (?Y , ¦ÒY2 )

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

2 / 22

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

3 / 22

6.5

Conditional Distributions

6.5

General Bivariate Normal - RNG

Multivariate Change of Variables

Consequently, if we want to generate a Bivariate Normal random variable

with X ¡« N (?X , ¦ÒX2 ) and Y ¡« N (?Y , ¦ÒY2 ) where the correlation of X and

Y is ¦Ñ we can generate two independent unit normals Z1 and Z2 and use

the transformation:

Let X1 , . . . , Xn have a continuous joint distribution with pdf f defined of S. We can define n

new random variables Y1 , . . . , Yn as follows:

¡¤¡¤¡¤

Y1 = r1 (X1 , . . . , Xn )

Then the joint pdf g of Y1 , . . . , Yn is

(

g (y1 , . . . , yn ) =

We can also use this result to find the joint density of the Bivariate

Normal using a 2d change of variables.

¡¤¡¤¡¤

6.5

for (y1 , . . . , yn ) ¡Ê T

otherwise

Where

?

J = det ?

?

April 11, 2012

xn = sn (y1 , . . . , yn )

f (s1 , . . . , sn )|J|

0

?

Lecture 22

Yn = rn (X1 , . . . , Xn )

If we assume that the n functions r1 , . . . , rn define a one-to-one differentiable transformation

from S to T then let the inverse of this transformation be

x1 = s1 (y1 , . . . , yn )

X = ¦ÒX Z1 + ? X

p

Y = ¦ÒY [¦ÑZ1 + 1 ? ¦Ñ2 Z2 ] + ?Y

Statistics 104 (Colin Rundel)

Conditional Distributions

4 / 22

?s1

?y1

¡¤¡¤¡¤

?s1

?yn

?

.

..

..

.

..

?

?

?

?sn

?y1

Statistics 104 (Colin Rundel)

.

¡¤¡¤¡¤

?sn

?yn

Lecture 22

Conditional Distributions

6.5

April 11, 2012

Conditional Distributions

General Bivariate Normal - Density

General Bivariate Normal - Density

The first thing we need to find are the inverses of the transformation. If

x = r1 (z1 , z2 ) and y = r2 (z1 , z2 ) we need to find functions h1 and h2 such

that Z1 = s1 (X , Y ) and Z2 = s2 (X , Y ).

Next we calculate the Jacobian,

X = ¦ÒX Z1 + ?X

Z1 =

=

1

s2 (x, y ) = p

1 ? ¦Ñ2



1

p

2¦Ð¦ÒX ¦ÒY

=

Lecture 22

#

"

= det

1

¦ÒX

?¦Ñ

¡Ì

¦ÒX 1?¦Ñ2

#

0

¡Ì1

¦ÒY

=

¦ÒX ¦ÒY

1?¦Ñ2

1

p

1 ? ¦Ñ2

f (x, y ) = f (z1 , z2 )|J|









1

1

1 2

1 2

2

2

=

exp ? (z1 + z2 ) |J| =

exp ? (z1 + z2 )

p

2¦Ð

2

2

2¦Ð¦ÒX ¦ÒY 1 ? ¦Ñ2

Therefore,

Statistics 104 (Colin Rundel)

?s1

?y

?s2

?y

The joint density of X and Y is then given by

X ? ?X

¦ÒX

p

Y = ¦ÒY [¦ÑZ1 + 1 ? ¦Ñ2 Z2 ] + ?Y

p

Y ? ?Y

X ? ?X

=¦Ñ

+ 1 ? ¦Ñ 2 Z2

¦ÒY

¦ÒX





1

Y ? ?Y

X ? ?X

Z2 = p

?¦Ñ

¦ÒY

¦ÒX

1 ? ¦Ñ2

x ? ?X

s1 (x, y ) =

¦ÒX

?s1

?x

?s2

?x

"

J = det

5 / 22

y ? ?Y

x ? ?X

?¦Ñ

¦ÒY

¦ÒX

"

1 ? ¦Ñ2

exp ?

¦Ñ2 )1/2

exp

"

x ? ?X

2

"

1

2¦Ð¦ÒX ¦ÒY (1 ?

1

¦ÒX

?1

2(1 ?

¦Ñ2 )

!2

+

1

y ? ?Y

1 ? ¦Ñ2

¦ÒY

(x ? ?X )2

2

¦ÒX

+

(y ? ?Y )2

2

¦ÒY

?¦Ñ

x ? ?X

? 2¦Ñ

!2 ##

¦ÒX

(x ? ?X ) (y ? ?Y )

¦ÒX

!#

¦ÒY



April 11, 2012

6 / 22

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

7 / 22

6.5

Conditional Distributions

6.5

Conditional Distributions

General Bivariate Normal - Density (Matrix Notation)

General Bivariate Normal - Density (Matrix Notation)

Obviously, the density for the Bivariate Normal is ugly, and it only gets

worse when we consider higher dimensional joint densities of normals. We

can write the density in a more compact form using matrix notation,







 



¦ÒX2

¦Ñ¦ÒX ¦ÒY

x

?X

¦²=

x=

?=

¦Ñ¦ÒX ¦ÒY

¦ÒY2

?Y

y

Recall for a 2 ¡Á 2 matrix,





1

1

?1/2

T ?1

f (x) =

(det ¦²)

exp ? (x ? ?) ¦² (x ? ?)

2¦Ð

2

Statistics 104 (Colin Rundel)


?1/2

=

6.5

b

d



A?1 =

1

det A



d

?c

?b

a



=

1

ad ? bc



d

?c

?b

a



Then,

(x ? ?)T ¦²?1 (x ? ?)





T 



1

x ? ?x

¦ÒY2

x ? ?x

?¦Ñ¦ÒX ¦ÒY

= 2 2

2

y ? ?y

y ? ?y

?¦Ñ¦ÒX ¦ÒY

¦ÒX

¦ÒX ¦ÒY (1 ? ¦Ñ2 )





T 

2

1

¦ÒY (x ? ?X ) ? ¦Ñ¦ÒX ¦ÒY (y ? ?Y )

x ? ?x

= 2 2

2

2

y ? ?y

?¦Ñ¦ÒX ¦ÒY (x ? ?X ) + ¦ÒX (y ? ?Y )

¦ÒX ¦ÒY (1 ? ¦Ñ )

=

1

¦ÒX ¦ÒY (1 ? ¦Ñ2 )1/2

Lecture 22

a

c




1

¦Ò 2 (x ? ?X )2 ? 2¦Ñ¦ÒX ¦ÒY (x ? ?X )(y ? ?Y ) + ¦ÒX2 (y ? ?Y )2

¦ÒX2 ¦ÒY2 (1 ? ¦Ñ2 ) Y

!

1

(x ? ?X )2

(x ? ?X )(y ? ?Y )

(y ? ?Y )2

=

?

2¦Ñ

+

1 ? ¦Ñ2

¦ÒX2

¦ÒX ¦ÒY

¦ÒY2

We can confirm our results by checking the value of (det ¦²)?1/2 and

(x ? ?)T ¦²?1 (x ? ?) for the bivariate case.

(det ¦²)?1/2 = ¦ÒX2 ¦ÒY2 ? ¦Ñ2 ¦ÒX2 ¦ÒY2



A=

April 11, 2012

8 / 22

Statistics 104 (Colin Rundel)

Conditional Distributions

Lecture 22

6.5

General Bivariate Normal - Examples

April 11, 2012

9 / 22

Conditional Distributions

General Bivariate Normal - Examples

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 2), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 2)

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 1)

¦Ñ=0

¦Ñ=0

¦Ñ=0

¦Ñ = 0.25

¦Ñ = 0.5

¦Ñ = 0.75

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

10 / 22

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

11 / 22

6.5

Conditional Distributions

6.5

General Bivariate Normal - Examples

Conditional Distributions

General Bivariate Normal - Examples

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 1)

X ¡« N (0, 2), Y ¡« N (0, 1)

X ¡« N (0, 1), Y ¡« N (0, 2)

¦Ñ = ?0.25

¦Ñ = ?0.5

¦Ñ = ?0.75

¦Ñ = ?0.75

¦Ñ = ?0.75

¦Ñ = ?0.75

Lecture 22

April 11, 2012

Lecture 22

April 11, 2012

Statistics 104 (Colin Rundel)

6.5

12 / 22

Statistics 104 (Colin Rundel)

Conditional Distributions

6.5

13 / 22

Conditional Distributions

Multivariate Normal Distribution

Multivariate Normal Distribution - Cholesky

Matrix notation allows us to easily express the density of the multivariate

normal distribution for an arbitrary number of dimensions. We express the

k-dimensional multivariate normal distribution as follows,

In the bivariate case, we had a nice transformation such that we could

generate two independent unit normal values and transform them into a

sample from an arbitrary bivariate normal distribution.

X ¡« Nk (?, ¦²)

There is a similar method for the multivariate normal distribution that

takes advantage of the Cholesky decomposition of the covariance matrix.

where ? is the k ¡Á 1 column vector of means and ¦² is the k ¡Á k

covariance matrix where {¦²}i,j = Cov (Xi , Xj ).

The Cholesky decomposition is defined for a symmetric, positive definite

matrix X as

L = Chol(X)

The density of the distribution is

where L is a lower triangular matrix such that LLT = X.





1

1

?1/2

T ?1

(det ¦²)

exp ? (x ? ?) ¦² (x ? ?)

f (x) =

2

(2¦Ð)k/2

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

14 / 22

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

15 / 22

6.5

Conditional Distributions

6.5

Conditional Distributions

Multivariate Normal Distribution - RNG

Cholesky and the Bivariate Transformation

Let Z1 , . . . , Zk ¡« N (0, 1) and Z = (Z1 , . . . , Zk )T then

We need to find the Cholesky decomposition of ¦² for the general bivariate

case where





2

¦ÒX

¦Ñ¦ÒX ¦ÒY

¦²=

? + Chol(¦²)Z ¡« Nk (?, ¦²)

this is offered without proof in the general k-dimensional case but we can

check that this results in the same transformation we started with in the

bivariate case and should justify how we knew to use that particular

transformation.

¦Ñ¦ÒX ¦ÒY

¦ÒY2

We need to solve the following for a, b, c



a

b

0

c



a

0

b

c





=

a2

ab

b2

ab

+ c2





=

¦ÒX2

¦Ñ¦ÒX ¦ÒY

¦Ñ¦ÒX ¦ÒY

¦ÒY2



This gives us three (unique) equations and three unknowns to solve for,

a2 = ¦ÒX2

ab = ¦Ñ¦ÒX ¦ÒY

b 2 + c 2 = ¦ÒY2

a = ¦ÒX

b = ¦Ñ¦ÒX ¦ÒY /a = ¦Ñ¦ÒY

q

c = ¦ÒY2 ? b 2 = ¦ÒY (1 ? ¦Ñ2 )1/2

Statistics 104 (Colin Rundel)

Lecture 22

6.5

April 11, 2012

16 / 22

Statistics 104 (Colin Rundel)

Conditional Distributions

Lecture 22

6.5

April 11, 2012

Conditional Distributions

Cholesky and the Bivariate Transformation

Conditional Expectation of the Bivariate Normal

Let Z1 , Z2 ¡« N (0, 1) then

Using X = ?X + ¦ÒX Z1 and Y = ?Y + ¦ÒY [¦ÑZ1 + (1 ? ¦Ñ2 )1/2 Z2 ] where

Z1 , Z2 ¡« N (0, 1) we can find E (Y |X ).



X

Y

17 / 22



= ? + Chol(¦²)Z



 





¦ÒX

0

?X

Z1

=

+

?Y

Z2

¦Ñ¦ÒY ¦ÒY (1 ? ¦Ñ2 )1/2



 



¦ÒX Z1

?X

=

+

?Y

¦Ñ¦ÒY Z1 + ¦ÒY (1 ? ¦Ñ2 )1/2 Z2

i

h





E [Y |X = x] = E ?Y + ¦ÒY ¦ÑZ1 + (1 ? ¦Ñ2 )1/2 Z2 X = x









x ? ?X

2 1/2

= E ?Y + ¦Ò Y ¦Ñ

+ (1 ? ¦Ñ ) Z2 X = x

¦ÒX





x ? ?X

2 1/2

= ?Y + ¦Ò Y ¦Ñ

+ (1 ? ¦Ñ ) E [Z2 |X = x]

¦ÒX





x ? ?X

= ?Y + ¦Ò Y ¦Ñ

¦ÒX

X = ?X + ¦Ò X Z 1

Y = ?Y + ¦ÒY [¦ÑZ1 + (1 ? ¦Ñ2 )1/2 Z2 ]

By symmetry,



E [X |Y = y ] = ?X + ¦ÒX ¦Ñ

Statistics 104 (Colin Rundel)

Lecture 22

April 11, 2012

18 / 22

Statistics 104 (Colin Rundel)

Lecture 22

y ? ?Y

¦ÒY



April 11, 2012

19 / 22

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download