__ PROBABILITY



OR 201.1 ELEMENTS OF PROBABILITY

Notes to accompany the lectures

Basic Probability

Conditional Probability

Discrete Probability Distributions

Properties of Discrete Distributions

Continuous Probability Distributions

The Normal Distribution

Chapter 1 PROBABILITY

1.1 Introduction

The basic tool on which statistical methods are based is PROBABILITY, and so it is necessary to spend some time understanding this important concept and its properties. The subject of probability is vast and volumes have been written about it. It can be treated in many ways - as an abstract mathematical topic developed from an axiomatic approach, or from intuitive sampling ideas in practice. In this Chapter we shall adopt the latter approach and deal with the basic properties of probability and their application. At this stage some of the illustrative examples will be simple and somewhat artificial; this is intentional as we do not wish to confuse the understanding of these concepts by introducing them in complex situations.

1.2 The Meaning of Probability

We are used to making "every day" statements regarding the likelihood of it raining or a football team winning a match; in these cases we are using probability in a qualitative fashion, in order to express our DEGREE OF BELIEF in an event's chances of occurring. To use this on a scientific basis, however, we need a more formal and exact concept of probability; we need to quantify our degree of belief.

Conventionally probability is defined to lie on a scale from 0 to 1 although this can be interpreted as a percentage from 0 to 100. Probability is a number associated with an event (A say) and lying between 0 and 1 so that:-

p(A) = 0 means that the event is impossible *

p(A) = 1 means that the event is certain *

p(A) > p(B) means that A is more likely to happen than B

Note that the p(.) notation for "the probability of ..." - this notation is very useful.

We thus have a probability scale.

--------------------------------------------------------------

* Although this is sufficient for our understanding at this stage, and for a practical view of probability, we shall see later, when dealing with continuous valued random variables that there is an exception to this.

1.3 Quantifying Probability

Although we are now able to rank events in this way it is not obvious how to actually quantify the probability of an event. For some simple situations this is quite straightforward. When an "experiment" can result in a number of equally likely basic events then the probability of the event of interest can be computed as:

Let N be the total number of equally likely simple events.

Let n be the number of these that are favourable to our event of interest.

Then the probability of the event is n/N.

Examples

A fair coin is tossed once and we are interested in the probability of heads turning up.

Then N = 2 corresponding to the two equally likely results, heads or tails.

Then n = 1 corresponding to heads.

So that the p of heads is equal to n/N = 1/2.

----------------------------

A fair die is tossed once and we are interested in the probability of a 6 turning up.

Then N = 6 corresponding to the six equally likely faces of the die.

Then n = 1 corresponding to a 6.

So that the p of a 6 is equal to n/N = 1/6.

Two fair dice are thrown. The number of possible equally likely outcomes can be represented diagrammatically as shown below. This set of points is called the sample space and a diagram of this sort is called a Venn diagram.

Sample Space:-

Red Die

1 2 3 4 5 6

1 X X X X X X

2 X X X X X X

Blue Die 3 X X X X X X

4 X X X X X X

5 X X X X X X

6 X X X X X X

Total number of points = N = 36

Probability that the sum = 5: n = 4; p = 4/36 = 1/9

Probability that the difference = 2: n = 8; p = 8/36 = 2/9

----------------------------------

The Venn Diagram is a very useful way of portraying the total basic results of an "experiment" and grouping them together to represent particular compound events of interest. It can be seen from this that there is a close relationship between probability theory and set theory.

Combinatorial Theory

It is clear from the above examples that counting the possible outcomes of an "experiment" is vital to probability determination; a useful tool to aid such counting is combinatorial theory and we summarize some basic useful formulae here:-

Factorials

The number of different ways that n different objects can be placed in order is n! This symbol, n!, is known as FACTORIAL n and is equal to,

n! = n*(n-1)*(n-2)*...3*2*1

eg 3! = 6 5! = 120 12! = 479,001,600

As well as this we define 0! = 1 in order to have compatibility with the formulae below.

Permutations

The number of different groups of r objects that can be chosen IN ORDER from n different objects is called the PERMUTATION of r from n.

It is equal to, nPr = n!/(n-r)!

For example, the number of different queues (lines) of 4 people that we can form from 6 people is 6P4 = 6!/2! = 360

Combinations

The number of different groups of r objects that can be chosen from n different objects (order immaterial) is called the COMBINATION of r from n.

It is equal to, nCr = n!/{r!(n-r)!}

For example, the number of different groups of 4 people that we can form from 6 people is 6C4 = 6!/(4!*2!) = 15

Using combinatorials to determine large counts is particularly useful in dealing with probabilities for card hands. It is also relevant to the Binomial and Poisson distributions that we shall study later.

Example A poker hand of 5 randomly chosen cards is dealt from a standard deck of 52 playing cards. We wish to determine the probability of "4 of a kind"; this means a hand such as 4 8's and any other.

Total number of possible hands, N = 52C5 = 2,598,960

Total number of ways of getting 4 cards of the same kind = 13

Total number of ways of getting the other card = 48

Hence n = 13*48 = 624

Then p(4 of a kind) = 624/2598960 = .00024 or .024%

1.5 Relative Frequency

The above techniques for determining probability apply only to situations which involve equally likely events. To apply probability in a wider context we need a more general interpretation and this is provided by the RELATIVE FREQUENCY interpretation of probability.

Imagine an experiment in which an event A either occurs or does not occur, and imagine F such experiments being conducted independently of each other. Suppose that A occurs f times. Then we may regard probability as the "settling down" value (or limit) of the relative frequency, f/F, as F becomes very large (-> (). Intuitively this is an appealingly interpretation and is extremely useful when we come to its use in statistical inference later on. Please note that we are not advocating that you should carry out all these experiments; they are imaginary, but the concept gives a meaning to numerical probability.

There is another, quite compatible view - probability is a degree of belief in the occurrence of an event and is based on the observer's knowledge of the situation. In other words 2 users with different states of knowledge can have different probabilities and both be correct; there is no absolute probability. It is a relationship between an observer and a situation. After all, if everyone had the same belief then the gambling industry would not thrive as it does today!

However 2 persons, with the same state of knowledge, and observing the same experimental results should have the same probabilities, and this is the basis for the use of probability as a scientific and objective tool. This will be our approach on this course.

1.6 The Meaning of Randomness

Statistical inference is concerned with the drawing of conclusions from data that are subject to randomness, perhaps due to the sampling procedure, or perhaps due to observational errors. Let us stop and think why, when we repeat an experiment under apparently identical conditions, we get different results. The answer is that although the conditions may be as identical as we are able to control them to be, there will inevitably be a large number of uncontrollable and unknown variables that we do not measure and which have a cumulative effect on the result of the sample or experiment. Their cumulative effect, therefore, is to is to cause variation in our results. It is this variation that we call RANDOMNESS and, although we never know the true generating mechanism for our data, we can take the random component into account via the concept of probability; this is why probability plays such an important role in data analysis.

1.7 The Properties of Probability

The Axioms

The whole of the mathematical theory of probability can be derived from a few basic axioms. These are:-

Probability is a number lying between 0 and 1

If 2 events cannot both happen (exclusive) as the result of an experiment then the probability of one of them happening is equal to the sum of their probabilities.

The total probability for all the possible exclusive events is 1.

In order to derive the probabilities of more complicated events from the knowledge of the probabilities of simpler events we need to know some fundamental laws that arise directly from the axioms. We summarize two of the most important here.

The Additive Law

Let A and B be any 2 events. Then:-

p(A and/or B) = p(A) + p(B) - p(both A and B)

In set notation:- p(A U B) = p(A) + p(B) - p(A ( B)

This is often referred to as the additive law of probability. The p(A U B) is the probability that at least 1 of the 2 events occurs, and p(A ( B) is the probability that both events occur. The correction term, - p(A ( B) is often not appreciated by the layman. For example, a person attempting a crossword competition once proposed that if his chance of winning was .001, and he entered 1000 competitions, then he would be certain to win! What would happen if he entered 2000 competitions!?

This law is best explained via a Venn Diagram. In Figure 1 we have the following representations:-

Event A is composed of all points in the left hand ellipse.

Event B is composed of all points in the right hand ellipse.

From the diagram:-

p(A) = area 1 + area 3

p(B) = area 2 + area 3

p(A ( B) = area 3

p(A U B) = area 1 + area 2 + area 3

Hence p(A U B) = p(A) + p(B) - p(A ( B)

[pic]

Figure 1

Example

Consider an industrial situation in which a machine component can be defective in 2 ways and such that:-

p(defective in first way) = .01

p(defective in second way) = .05

p(defective in both ways) = .001

Then it follows that the probability that the component is defective is

.01 + .05 - .001 = .059

Note that the probability of it being defective in both ways is not

.01 * .05 as we might expect. We shall explain this in section 2.6 below.

Two important cases of the additive law are worth noting.

1. If A and B cannot both happen then they are called EXCLUSIVE events and the probability of 1 of them happening is the sum of their individual probabilities.

2. The probability of an event NOT happening is 1 minus the probability of its happening.

p(Ac) = 1 - p(A)

The Multiplicative Law

This rule is concerned with the probability of 2 events happening at the same time. If the events are INDEPENDENT - ie they have no influence on each other - then the probability of both happening is the product of their individual probabilities.

p(A ( B) = p(A) * p(B)

This is a simple law and easy to accept intuitively. If we toss 2 fair coins then the probability of heads on both is .5 * .5 = .25. This product law does not hold for dependent events however; this is the subject of conditional probability and Bayes' Theorem which is discussed in the next section.

When meeting this subject for the first time there is sometimes some confusion of the terms "exclusive" and "independent". They have totally different meanings and are quite incompatible in the sense that 2 events cannot be exclusive and independent. If 2 events are exclusive then the occurrence of one denies the occurrence of the other, and so has a very strong effect on it - they are certainly not independent.

-----------------------

1

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download