Cumulative distribution function (cdf)

Stat 463/853-2020 Lecture 10

09.30.20

The method of cdf

Cumulative distribution function (cdf ) can be used for solving different problems. One of its most useful applications is in finding distribution of transformed random variables. We will illustrate this method by several examples.

Example 1. Let X U[0, 1] be a uniformly distributed random variable, with corresponding pdf fX(x) and cdf FX(x). Suppose we are interested in the distribution of the transformed random variable Y = X2. The method of cdf is very simple and efficient in such problems.

First, note that 0 Y 1, so we know already that fY (y) = 0 for y outside [0, 1]. The values of fY (y) at the points 0 and 1 do not matter! Now, for 0 < y < 1,

FY (y) = P(Y y) = P(X2 y) = P(X y1/2) = FX (y1/2) = y1/2.

Thus,

fY (y)

=

FY (y)

=

1 y-1/2, 2

0 < y < 1.

These are the cdf and the pdf of Y = X2. Note that fY (y) has an asymptote at the point y = 0.

Example

2.

Let

again

X

U [0, 1],

and

let

Y

=

-

1

ln

X

0, for some parameter

> 0. Here for y 0,

1

1

FY (y) = P(Y y) = P - ln X y = P ln X -y = P(ln X -y) =

make yourself a picture of the lon !

P(e-y X 1) = FX (1) - FX (e-y) = 1 - e-y, y > 0,

while

FX(y) = 0, y 0.

Thus,

fY (y) = FY (y) =

e-y 0

, ,

y > 0, y 0.

We have found that here Y has exponential distribution, with a parameter > 0. Observe, that an exponential random variable Y E() can be thus obtained by transforming a uniform random variable X. (In fact, any random variable can be obtained by some transformation of a uniform random variable X!)

Example 3. The `ab'-formula. Let X be a continuous random variable, X fX(x), FX(x),

1

and suppose we are interested in the distribution of a linear function Y = a+bX (b > 0).

We start by evaluating the cdf of Y :

y-a

y-a

FY (y) = P(Y y) = P(a + bX y) = P X b = FX b .

Next, applying the chain rule for differentiation of a composite function,

fY (y) = FY (y) = FX

y-a b

11 ? b = b fX

y-a b

.

This is a very useful formula, which will be used on a number of occasions. We will refer to it as `ab' formula, although this is not a standard terminology (so don't look for it on the internet ? you won't find it there!)

Example 4. Let again X U [0, 1], and let Y = a + (b - a)X, for some a < b. By the `ab' formula,

1 fY (y) = b - a fX

y-a b-a

=

0

1

b-a

0

, , ,

y b.

y-a FY (y) = FX b - a

=

0,

y-a b-a

,

1,

y b.

We say that a random variable with such pdf and cdf has the uniform distribution, on the interval [a, b]. The shorthand notation: Y U [a, b].

Jointly continuous distributions

Recall that a random variables X is said to have a continuous distribution, with a pdf fX(x), if for any a < b, finite or infinite,

b

P(a X b) = fX(x) dx.

a

Of course,

f (x) 0,

f (x) dx = 1.

-

Definition. Random variables X and Y have jointly continuous distribution,

with a joint pdf f (x, y), if for any a < b and c < d, finite or infinite,

bd

P(a X b, c Y d) =

f (x, y) dy dx.

ac

Of course, here

f (x, y) 0,

f (x, y) dy dx = 1

- -

2

If X and Y have a joint pdf f (x, y), then for any region G, with a piece-wise smooth boundary,

P((X, Y ) G) =

f (x, y) dy dx.

G

Such probability has the same properties as area, weight, and other measures. A good example is

S(G) = Area(G) =

1 dx dy.

G

In most cases, calculation of double integrals, like the last two, is reduced to a

repeated applications of ordinary one-dimensional integrals. However, one has to

be mindful of the limits of integration in the latter integrals. Let us illustrate this

by the following simple example.

Example. Suppose we are interested in the area of the following region G = {0

x 1, 0 y 1 - x} (triangle). We have (it may help to make a picture of

the area of integration)

S(G) =

1

1-x

1

(1 - x)2 1

11

dx dy = dx

dy = (1-x) dx = -

=- 0- = .

G

0

0

0

20

22

Of course, we can also calculate the area by reversing the order of integration:

1

1-y

S(G) = dy

dx.

0

0

Verify yourself, that the result will be the same.

Let fX(x) and fY (y) be the marginal pdf 's of X and Y , respectively. As in the discrete case, we have

The elimination rule.

fX(x) = f (x, y) dy, fY (y) = f (x, y) dx.

-

-

Here is the idea of the proof. For any x,

x

x

fX(u) du = P(X x) = P(X x, - < Y < ) =

f (u, y) dy du.

-

- -

At each point where fX(x) is continuous, the left-hand side is differentiable w.r.t.

x; hence, so is the right-hand side. Since both sides are equal, for all x, so

are their derivatives! By differentiating both sides w.r.t. the upper limit x,

one gets

fX(x) = f (x, y) dy.

2.

-

3

Independence In the discrete case, X and Y were called independent, if their joint pmf could be factorized, for all xi and yj, as

f (xi, yj) = P(X = xi, Y = yj) = P(X = xi)P(Y = yj) = fX (xi)fY (yj).

In the continuous case there are 2 equivalent definitions: 1: X and Y are independent, if their joint pdf factorizes as

f (x, y) = fX(x)fY (y).

(1)

2. X and Y are independent, if for any "good" subsets A, B, like finite or infinite

intervals,

P(X A, Y B) = P(X A)P(Y B).

(2)

Let us prove, first, that property 1 implies 2. By (1), we have

P(X A, Y B) =

f (x, y) dy dx =

fX(x)fY (y) dy dx =

AB

AB

fX(x) fY (y) dy dx = fX(x) dx

A

B

A

fY (y) dy = P(X A)?P(Y B).

B

Now, let us prove that property 2 implies 1. Assuming (2) , and using the last line,

we get, for all a, b,

a

b

P(X a, Y b) =

f (x, y) dy dx =

- -

P(X a) ? P(Y b) =

a

fX(x) dx)

-

b

fY (y) dy .

-

Since the integrals on both sides are equal, so are their derivatives, with respect to

a. Thus, differentiating the integrals with respect to the upper limit a, we get, for

any b,

b

b

f (a, y) dy = fX(a) fY (y) dy.

-

-

Now, now let us differentiate both sides of this equality with respect to b. We will

get then, for any a and b,

f (a, b) = fX(a)fY (b).

Definition. Random variables X, Y, ...Z have a jointly continuous distribution, with a pdf f (x, y, ..., z), if for and a < b, c < d, ..., u < v,

v

db

P(a X b, c Y d, ..., u Z v) = ? ? ?

f (x, y, ..., z) dxdy ? ? ? dz.

u

ca

Definition. Continuous random variables X, Y, ...Z are independent, if

f (x, y, ..., z) = fX(x)fY (y) ? ? ? fZ(z).

4

Conditional distribution and conditional expectation

Recall that in the discrete case, for every y, such that fY (y) > 0, the conditional pmf of X, given Y = y, was defined as

f (x, y)

f (x|y) = P(X = x|Y = y) =

.

fY (y)

Similarly, in the continuous case, the conditional pdf of X, given Y = y, is defined, for all y such that fY (y) > 0, as

f (x, y)

f (x|y) =

.

(3)

fY (y)

Of course, if fY (y) > 0, then by the elimination rule,

1 f (x|y) dx =

f (x, y) dx = fY (y) = 1.

fY (y)

fY (y)

In other words, a conditional pdf is a pdf, and therefore, it has the same properties as all other pdf's. In particular, if fY (y) > 0, the conditional expectation

E(g(X)|Y = y) = g(x)f (x|y) dx.

has all the usual properties of expectations. From (3), we get the multiplication rule,

f (x, y) = f (x|y)fY (y).

From the elimination and multiplication rules, we get the so-called Law of total probability,

fX(x) = f (x, y) dy = f (x|y)fY (y) du;

and the Bayes theorem:

f (x, y)

f (y|x) =

=

f (x|y)fY (y) .

fX (x)

f (x|u)fY (u) du

If jointly continuous random variables X and Y are independent, then

f (x|y)

=

f (x, y) fY (y)

=

fX (x)fY (y) fY (y)

=

fX (x).

Thus, if X and Y are independent, the conditional pdf f (x|y) does not depend on y. To summarize: we have two criteria of the dependence: if either f (x, y) = fX(x)fY (y) or, if the conditional pdf f (x|y) depends on y, then X and Y are dependent!

5

Mixed distributions

It may well happen that one of the variables, say X, is discrete, while another, Y , is continuous. In such cases f (x, y) is a mixed pmf/pdf: discrete w.r.t. x, continuous w.r.t. y. Usually, such cases do not cause technical difficulties, but they may lead to some unexpected results. Let us illustrate this by the following

Example. Assume that P U [0, 1] is a uniform random variable, with the pdf

fP (p) = 1, 0 p 1.

Suppose next that, given P = p, X is a Bernoulli random variable B(p). In other words, the conditional pmf of X, given P = p, satisfies

f (x|p) = px(1 - p)1-x, x = 0, 1, 0 p 1.

Thus, X and P jointly have a mixed distribution, discrete for X, and continuous for P . Next, according to the multiplication rule, their joint pmf/pdf satisfies

f (x, p) = f (x|p)fP (p) = px(1 - p)1-x ? 1 = px(1 - p)1-x, x = 0, 1, 0 p 1. (4)

Recall next the beta integral from the previous lecture:

B(, ) =

1

p-1(1

-

p)-1

dp

=

()() ,

0

( + )

and the corresponding beta pdf,

B(p; , ) = ( + ) p-1(1 - p)-1, 0 p 1. ()()

It is not difficult to find the expectation of the beta random variable:

( + )

1

p ? p-1(1 - p)-1 dp =

.

(5)

()() 0

+

Let us now find the marginal pmf of X:

1

1

fX(x) = f (x, p) dp = px(1 - p)1-x dp =

0

0

1

p(x+1)-1(1 - p)(2-x)-1 dp =

(x + 1)(2 - x)

=

0

((x + 1) + (2 - x))

(x + 1)(2 - x) = (x + 1)(2 - x) =

(3)

2

1 2

,

1 2

,

x = 0, x = 1.

(6)

Thus, by (4) and (6),

f (x, p)

f (p|x) =

=

(3)

p(x+1)-1(1 - p)(2-x)-1 B(p; x + 1, 2 - x).

fX(x) (x + 1)(2 - x)

= =

Suppose we don't know the `true' value of P , but we are given the value of X = x. Then, our "best guess" for P is given by the conditional expectation of P , given X = x. According to (5), it equals

1

x+1

E(P |X = x) = pf (p|x) dp =

=

=

0

+ 3

1 3

,

2 3

,

x = 0, x = 1.

6

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download