Notes on Probability Theory and Statistics

[Pages:138]Notes on Probability Theory and Statistics

Antonis Demos (Athens University of Economics and Business)

October 2002

2

Part I Probability Theory

3

Chapter 1

INTRODUCTION

1.1 Set Theory Digression

A set is defined as any collection of objects, which are called points or elements. The biggest possible collection of points under consideration is called the space, universe, or universal set. For Probability Theory the space is called the sample space.

A set A is called a subset of B (we write A B or B A) if every element of A is also an element of B. A is called a proper subset of B (we write A B or B A) if every element of A is also an element of B and there is at least one element of B which does not belong to A.

Two sets A and B are called equivalent sets or equal sets (we write A = B) if A B and B A.

If a set has no points, it will be called the empty or null set and denoted by .

The complement of a set A with respect to the space , denoted by A?, Ac, or - A, is the set of all points that are in but not in A.

The intersection of two sets A and B is a set that consists of the common elements of the two sets and it is denoted by A B or AB.

The union of two sets A and B is a set that consists of all points that are in A or B or both (but only once) and it is denoted by A B.

The set difference of two sets A and B is a set that consists of all points in

6

Introduction

A that are not in B and it is denoted by A - B.

Properties of Set Operations

Commutative: A B = B A and A B = B A. Associative: A(B C) = (A B)C and A(B C) = (A B)C. Distributive: A (B C) = (A B) (A C) and A (B C) = (A B) (A C). (Ac)c = ?A?? = A i.e. the complement of the A-complement is A. If A subset of (the space) then: A = A, A = , A = , A = A, A A = , A A = , A A = A, and A A = A. De Morgan Law: (A B) = A B, and (A B) = A B. Disjoint or mutually exclusive sets are the sets that their intersection is the empty set, i.e. A and B are mutually exclusive if A B = . Subsets A1, A2, .... are mutually exclusive if Ai Aj = for any i 6= j.

Uncertainty or variability are prevalent in many situations and it is the purpose of the probability theory to understand and quantify this notion. The basic situation is an experiment whose outcome is unknown before it takes place e.g., a) coin tossing, b) throwing a die, c) choosing at random a number from N, d) choosing at random a number from (0, 1).

The sample space is the collection or totality of all possible outcomes of a conceptual experiment. An event is a subset of the sample space. The class of all events associated with a given experiment is defined to be the event space.

Let us describe the sample space S, i.e. the set of all possible relevant outcomes of the above experiments, e.g., S = {H, T } , S = {1, 2, 3, 4, 5, 6} . In both of these examples we have a finite sample space. In example c) the sample space is a countable infinity whereas in d) it is an uncountable infinity.

Classical or a priori Probability: If a random experiment can result in N mutually exclusive and equally likely outcomes and if N(A) of these outcomes have an attribute A, then the probability of A is the fraction N(A)/N i.e. P (A) = N(A)/N,

Set Theory Digression

7

where N = N(A) + N(A).

Example: Consider the drawing an ace (event A) from a deck of 52 cards.

What is P (A)?

We have that N(A) = 4 and N(A) = 48. Then N = N(A) + N(A) = 4 + 48 =

52

and

P (A)

=

N (A) N

=

4 52

Frequency or a posteriori Probability: Is the ratio of the number that

an event A has occurred out of n trials, i.e. P (A) = /n.

Example: Assume that we flip a coin 1000 times and we observe 450 heads.

Then the a posteriori probability is P (A) = /n = 450/1000 = 0.45 (this is also the

relative frequency). Notice that the a priori probability is in this case 0.5.

Subjective Probability: This is based on intuition or judgment.

We shall be concerned with a priori probabilities. These probabilities involve,

many times, the counting of possible outcomes.

1.1.1 Some Counting Problems

Some more sophisticated discrete problems require counting techniques. For example:

a) What is the probability of getting four of a kind in a five card poker? b) What is the probability that two people in a classroom have the same birthday?

The sample space in both cases, although discrete, can be quite large and it not feasible to write out all possible outcomes.

1. Duplication is permissible and Order is important (Multiple Choice Arrangement), i.e. the element AA is permitted and AB is a different element from BA. In this case where we want to arrange n objects in x places the possible outcomes is given from: Mxn = nx.

Example: Find all possible combinations of the letters A, B, C, and D when duplication is allowed and order is important.

The result according to the formula is: n = 4, and x = 2, consequently the

8

Introduction

possible number of combinations is M24 = 42 = 16. To find the result we can also use a tree diagram.

2. Duplication is not permissible and Order is important (Per-

mutation Arrangement), i.e. the element AA is not permitted and AB is a

different element from BA. In this case where we want to permute n objects in x

places the possible outcomes is given from:

Pxn

or

P (n,

x)

=

n

?

(n

-

1)

?

..(n

-

x

+

1)

=

(n

n! .

- x)!

Example: Find all possible permutations of the letters A, B, C, and D when

duplication is not allowed and order is important.

The result according to the formula is: n = 4, and x = 2, consequently the

possible

number

of

combinations

is

P24

=

4! (4-2)!

=

234 2

=

12.

3. Duplication is not permissible and Order is not important

(Combination Arrangement), i.e. the element AA is not permitted and AB is

not a different element from BA. In this case where we want the combinations of n

objects in x places the possible outcomes is given from:

??

Cxn

or

C(n, x)

=

P (n, x) x!

=

n! (n - x)!x!

=

n x

Example: Find all possible combinations of the letters A, B, C, and D when

duplication is not allowed and order is not important.

The result according to the formula is: n = 4, and x = 2, consequently the

possible

number

of

combinations

is

C24

=

4! 2!(4-2)!

=

234 22

=

6.

Let us now define probability rigorously.

1.1.2 Definition of Probability

Consider a collection of sets A with index , which is denoted by {A : }. We can define for an index of arbitrary cardinality (the cardinal number of a set is the number of elements of this set):

A = {x S : x A for some }

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download