Lecture 10: Joint Distributions & Order Statistics

Lecture 10: Joint Distributions & Order Statistics

Sta 111

Colin Rundel

May 28, 2014

Joint Distributions Discrete RVs

Joint Distribution - Example, cont.

Let B be the number of Black socks and W the number of White socks drawn, then the joint distribution of B and W is given by:

W

012

0

1 8 6 15 66 66 66 66

B

1

12 66

24 66

0

36 66

2

15 66

0

0

15 66

28 32 6 66 66 66 66 66

1/66 8/66 6/66 12/66 24/66 P(B = b, W = w ) = 0/66 15/66 0/66 0/66 0

If b=0,w=0 If b=0,w=1 If b=0,w=2 If b=1,w=0 If b=1,w=1 If b=1,w=2 If b=2,w=0 If b=2,w=1 If b=2,w=2 otherwise

64

2

P(B = b, W = w ) =

b

w

2-b-w 12

, for 0 b, w 2 and b + w 2

2

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 2 / 40

Joint Distributions Discrete RVs

Joint Distribution - Example

Draw two socks at random, without replacement, from a drawer full of twelve colored socks:

6 black, 4 white, 2 purple

Let B be the number of Black socks, W the number of White socks drawn, then the distributions of B and W are given by:

0

1

2

P(B=k)

6 12

5 11

=

15 66

2

6 12

6 11

=

36 66

6 12

5 11

=

15 66

P(W=k)

8 12

7 11

=

28 66

2

4 12

8 11

=

32 66

4 12

3 11

=

6 66

Note - B HyperGeo(12, 6, 2) =

Sta 111 (Colin Rundel)

66 k 2-k

12 2

and W HyperGeo(12, 4, 2) =

Lecture 10

48 k 2-k

12 2

May 28, 2014

1 / 40

Joint Distributions Discrete RVs

Marginal Distributions

Note that the column and row sums are the distributions of B and W respectively.

P(B = b) = P(B = b, W = 0) + P(B = b, W = 1) + P(B = b, W = 2) P(W = w ) = P(B = 0, W = w ) + P(B = 1, W = w ) + P(B = 2, W = w )

These are the marginal distributions of B and W . In general,

P(X = x) = P(X = x, Y = y ) = P(X = x|Y = y )P(Y = y )

y

y

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 3 / 40

Joint Distributions Discrete RVs

Conditional Distribution

Conditional distributions are defined as we have seen previously with

P(X = x|Y = y ) = P(X = x, Y = y ) = joint pmf

P(Y = y )

marginal pmf

Therefore the pmf for white socks given no black socks were drawn is

1

P (W

=

w |B

=

0)

=

P(W = w , B = P(B = 0)

0)

=

66

8

66

6

66

15 66

=

1 15

15 66

=

8 15

15 66

=

6 15

if W = 0 if W = 1 if W = 2

Joint CDF

Joint Distributions Continuous RVs

F (x, y ) = P[X x, Y y ] = P[(X , Y ) lies south-west of the point (x, y )]

Y (x,y)

q

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 4 / 40

Joint CDF, cont.

Joint Distributions Continuous RVs

The joint Cumulative distribution function follows the same rules as the univariate CDF,

Univariate definition:

lim F (x) = 0

x -

x

F (x) = P(X x) =

f (z)dz

-

lim F (x) = 1

x

x y F (x) F (y )

Bivariate definition:

y

x

F (x, y ) = P(X x, Y y ) =

f (x, y ) dx dy

- -

lim F (x, y ) = 0

x,y - Sta 111 (Colin Rundel)

lim F (x, y ) = 1

x,y Lecture 10

x x ,y y F (x, y ) F (x , y )

May 28, 2014 6 / 40

Sta 111 (Colin Rundel)

X

Lecture 10

May 28, 2014 5 / 40

Marginal CDFs

Joint Distributions Continuous RVs

We can define marginal CDFs using the joint CDF by setting one of the values to infinity:

x

F (x, ) = P(X x, Y ) =

f (x, y ) dy dy

- -

= P(X x) = FX (x)

y

F (, y ) = P(X , Y y ) =

f (x, y ) dx dy

- -

= P(Y y ) = FY (y )

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 7 / 40

Joint pdf

Joint Distributions Continuous RVs

Similar to the CDF the probability density function follows the same general rules in two dimensions,

Univariate definition:

f (x) 0 for all x

f (x)

=

d dx

F

(x

)

-

f

(x )dx

=

1

Bivariate definition:

f (x, y ) 0 for all (x, y )

f (x, y ) =

F (x, y )

x y

f (x, y ) dx dy = 1

- -

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 8 / 40

Joint Distributions Continuous RVs

Probability and Expectation

Univariate definition:

P(X A) = f (x) dx

A

E [g (X )] =

g (x) ? f (x) dx

-

Bivariate definition:

P(X A, Y B) =

f (x, y ) dx dy

AB

E [g (X , Y )] =

g (x, y ) ? f (x, y ) dx dy

- -

Marginal pdfs

Joint Distributions Continuous RVs

Marginal pdfs are derived by integrating out one of the random variables.

fX (x) = fY (x) =

f (x, y ) dy

-

f (x, y ) dx

-

Previously we defined independence in terms of, X and Y are independent if and only if E (XY ) = E (X )E (Y ).

An equivalent definitions is, X and Y are independent if and only if f (x, y ) = fX (x)fY (y ).

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 9 / 40

Joint Distributions Continuous RVs

Example 1 - Joint Uniforms

Let f (x, y ) = c for x (0, 1), y (0, 1). Find c.

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 10 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 11 / 40

Example 1, cont

Joint Distributions Continuous RVs

Given the f (x, y ) we just found, find F (x, y ).

Example 1, cont.

Joint Distributions Continuous RVs

Verify that the F (x, y ) we just found gives the correct f (x, y ),

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 12 / 40

Example 1, cont.

Joint Distributions Continuous RVs

Check that f (x, y ) produces the correct marginal densities for X and Y (fX (x) and fY (y ))

Sta 111 (Colin Rundel)

Lecture 10

Example 1, cont.

Joint Distributions Continuous RVs

Find the expected value of X and Y

May 28, 2014 13 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 14 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 15 / 40

Example 1, cont.

Joint Distributions Continuous RVs

Find the expected value of XY

Example 2

Joint Distributions Continuous RVs

Let f (x, y ) = cx2y for x2 y 1.

Find: a) c b) P(|X | Y ) c) fX (x) and fY (y )

Sta 111 (Colin Rundel)

Lecture 10

Joint Distributions Continuous RVs

Example 2 - Range

1

May 28, 2014 16 / 40

Sta 111 (Colin Rundel)

Lecture 10

Example 2.a

Find c

Joint Distributions Continuous RVs

May 28, 2014 17 / 40

Y

-1 Sta 111 (Colin Rundel)

0 X

Lecture 10

1 May 28, 2014 18 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 19 / 40

Example 2 - pdf

Joint Distributions Continuous RVs

0.15

f(x,y) 0.10

0.05

0.00

1.0

0.8

1.0

0.6

0.5

0.4

y

0.2

0.0 -1.0

-0.5

0.0 x

Sta 111 (Colin Rundel)

Lecture 10

Joint Distributions Continuous RVs

Example 2.b, cont.

Example 2.b

Joint Distributions Continuous RVs

0.20

Find P(|X | Y ).

To do this we need to integrate over the region where x2 y 1 and

0.15

|x| y which is indicated in red below

0.10

1

1

0.05

Y

Y

0.00

May 28, 2014 20 / 40

-1

0

X

Sta 111 (Colin Rundel)

1

-1

Lecture 10

Example 2.c

Joint Distributions Continuous RVs

Find the marginal densities

0 X

May 28, 2014

1

21 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 22 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 23 / 40

Joint Distributions Continuous RVs

Example 2.c, cont.

It is always a good idea to check that the marginals are proper densities.

Example 3

Joint Distributions Continuous RVs

Let Y be the rate of calls at a help desk, and X the number of calls between 2 pm and 4 pm one day; Let's say that:

f (x, y ) = (2y )x e-3y x!

for y > 0, x = 0, 1, 2, . . ..

Find: a) P(X = 0) b) P(Y > 2)

Sta 111 (Colin Rundel)

Lecture 10

Example 3.a

Find P(X = 0)

Joint Distributions Continuous RVs

May 28, 2014 24 / 40

Sta 111 (Colin Rundel)

Lecture 10

Example 3.b

Find P(Y > 2)

Joint Distributions Continuous RVs

May 28, 2014 25 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 26 / 40

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 27 / 40

Order Statistics

Order Statistics

Let X1, X2, X3, X4, X5 be iid random variables with a distribution f on the range (a, b). We can relabel these X's such that their labels correspond to arranging them in increasing order

X(1) X(2) X(3) X(4) X(5).

X(1)

a

X5

X(2)

X(3)

X(4)

X1

X4

X2

X(5)

X3

b

In the case where the distribution f is continuous we can make the stronger statement

X(1) < X(2) < X(3) < X(4) < X(5). Since P(Xi = Xj ) = 0 for all i = j for continuous random variables.

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014

28 / 40

Notational Detour

Order Statistics

For a continuous random variable

f (x) P(x X x + ) = P(X [x, x + ]) f (x) P(x X x + ) = P(X [x, x + ])/

f (x) = lim P(X [x, x + ])/ 0

P (x X x + ) = P (X 2 [x, x + ])

Sta 111 (Colin Rundel)

f (x) f (x + )

x x+ Lecture 10

May 28, 2014 30 / 40

Order Statistics

Order Statistics, cont.

For X1, X2, . . . , Xn iid random variables X(k) is the kth smallest X , usually called the kth order statistic. The first order statistic, X(1) is therefore the smallest Xi and

X(1) = min(X1, . . . , Xn)

Similarly, the nth order statistic, X(n) is the largest Xi and X(n) = max(X1, . . . , Xn)

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 29 / 40

Order Statistics

Density of the maximum

For X1, X2, . . . , Xn iid continuous random variables with pdf f and cdf F the density of the maximum is

P(X(n) [x, x + ]) = P(one of the X 's [x, x + ] and all others < x)

n

= P(Xi [x, x + ] and all others < x)

i =1

= nP(X1 [x, x + ] and all others < x) = nP(X1 [x, x + ])P(all others < x) = nP(X1 [x, x + ])P(X2 < x) ? ? ? P(Xn < x) = nf (x) F (x)n-1

f(n)(x ) = nf (x )F (x )n-1

Sta 111 (Colin Rundel)

Lecture 10

May 28, 2014 31 / 40

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download