2.3 Simple Random Sampling - Montana State University

2.3 Simple Random Sampling

? Simple random sampling without replacement (srswor) of size n is the probability sampling design for which a fixed number of n units are selected from a population of N units without replacement such that every possible sample of n units has equal probability of being selected. A resulting sample is called a simple random sample or srs.

? Note: I will use SRS to denote a simple random sample and SR as an abbreviation of `simple random'.

? Some necessary combinatorial notation:

? (n factorial) n! = n ? (n - 1) ? (n - 2) ? ? ? ? ? 2 ? 1. This is the number of

unique arrangements or orderings (or permutations) of n distinct items. For example:

6! = 6 ? 5 ? 4 ? 3 ? 2 ? 1 = 720.

? (N choose n)

N n

=

N (N - 1) ? ? ? (N - n + 1) n!

=

N! n!(N - n)! .

This is the

number of combinations of n items selected from N distinct items (and the order of

selection doesn't matter). For example,

6 2

=

6! 2!4!

=

(6)(5)(4!) 2!4!

=

(6)(5) (2)(1)

=

15.

? There are

N n

possible SRSs of size n selected from a population of size N .

? For any SRS of size n from a population of size N , we have

P (S)

=

1/

N n

.

? Unless otherwise specified, we will assume sampling is without replacement.

2.3.1 Estimation of yU and t

? A natural estimator for the population mean yU is the sample mean y. Because y is an estimate of an individual unit's y-value, multiplication by the population size N will give us an estimate t of the population total t. That is:

yU

=y=

1 n

n

yi

i=1

Nn

t= n

yi =

i=1

(10)

? yU and t are design unbiased. That is, the average values of y and N y taken over all possible SRSs equal yU and t, respectively.

Demonstration of Unbiasedness: Suppose we have a population consisting of five y-values:

Unit i 1 2 3 4 5

yi

02347

which has the following parameters:

N=

t=

yU =

S2 =

S

Suppose

a

SRS

of

size

n=2

is

selected.

Then

P (S) =

1/

5 2

= 1/10 for each of the 10 possible

SRSs.

22

All Possible Samples and Statistics from Example Population

Sample S1 S2 S3 S4 S5 S6 S7 S8 S9 S10

Units 1,2 1,3 1,4 1,5 2,3 2,4 2,5 3,4 3,5 4,5

y-values 0,2 0,3 0,4 0,7 2,3 2,4 2,7 3,4 3,7 4,7

yi yU = y

2

1

3

1.5

4

2

7

3.5

5

2.5

6

3

9

4.5

7

3.5

10

5

11

5.5

t = Ny 5 7.5 10

17.5 12.5 15 22.5 17.5 25 27.5

S2 = s2 2 4.5 8

24.5 .5 2 12.5 .5 8 4.5

S=s 1.4142 2.1213 2.8284 4.9497 0.7071 1.4142 3.5355 0.7071 2.8284 2.1213

Column Sum

32

160

67

22.6274

Expected value = E(estimator)

32 10

=

3.2

160 10

=

16

67 10

=

6.7

22.6274 10

=

2.26274

= yU

=t

= S2

=S

The averages for estimators yU = y, t = N y, and S2 = s2 equal the parameters that they are estimating. This implies that y, N y, and s2 are unbiased estimators of yU , t, and S2.

Notation: E(yU ) = yU , E(t) = t, E(S2) = S2 or E(y) = yU , E(N y) = t, E(s2) = S2.

The average for estimator S = s does not equal the parameter S. This implies that s is a biased estimator of S. Notation: E(S) = S or E(s) = S.

? The next problem is to study the variances of yU = y and t = N y.

?

Warning: In mean V (Y )

an =

introductory statistics course, you S2/n (= 2/n) and its standard

were told deviation

that the variance of the is S/ n (= / n).

sample This is

appropriate if a sample was to be taken from an infinite or extremely large population.

? However, we are dealing with finite populations that often are not considered extremely large. In such cases, we have to adjust our variance formulas by N - n which is known N as the finite population correction (f.p.c.).

? Texts may rewrite the f.p.c. N - n as either 1 - n or 1 - f where f = n/N is the

N

N

fraction of the population that was sampled. By definition :

V (yU ) = V (y) =

V (t) = N 2V (y) = N (N - n) S2

(11)

n

? Because S2 is unknown, we use s2 to get unbiased estimators of the variances in (11)::

V (yU ) = V (y) =

V (t) = N 2V (y) = N (N - n) s2

(12)

n

? Taking a square root of a variance in (11) yields the standard deviation of the estimator.

? Taking a square root of an estimated variance in (12) yields the standard error of the estimate.

23

? Thus, V (y) =

N -n N

S2 n

=

3 6.7 52

=

and

V (t)

=

N 2V (y)

=

N (N - n) S2 n

=

(5)(3)

6.7 2

=

.

? Like yU and t, the variances V (yU ) and V (t) are design unbiased. That is the average

of V (yU ) and V (t) taken over all possible SRSs equal V (yU ) = 2.01 and V (t) = 50.25, respectively.

? For the estimated variances we have V (yU ) =

N -n N

s2 n

=

3 s2 52

=

and

V (t)

=

N (N - n) s2 n

=

(5)(3)

s2 2

=

where s2 is a particular sample variance.

Example: We will use our population from the previous example:

Unit, i 1 2 3 4 5

yi

02347

which have the following parameters

N = 5 t = 16 yU = 3.2 S2 = 6.7 S 2.588 Estimated Variances of yU and t for All Samples

Sample S1 S2 S3 S4 S5 S6 S7 S8 S9 S10

Units 1,2 1,3 1,4 1,5 2,3 2,4 2,5 3,4 3,5 4,5

y-values

s2

0,2

2

0,3

4.5

0,4

8

0,7

24.5

2,3

.5

2,4

2

2,7

12.5

3,4

.5

3,7

8

4,7

4.5

Column Sum

V (yU ) = .3s2 0.6 1.35 2.4 7.35 0.15 0.6 3.75 0.15 2.4 1.35

V (t) = 7.5s2 15

33.75 60

183.75 3.75 15 93.75 3.75 60 33.75

? From the table we have E(V (yU )) = 20.1/10 = 2.01 = V (yU ) and E(V (t)) = 502.5/10 = 50.25 = V (t). Thus, we see that both variance estimators are unbiased.

? If N is large relative to n, then the finite population correction (f.p.c.) will be close to (but less than) 1. Omitting the finite population correction from the variance formulas (i.e., replacing (N - n)/N with 1) will slightly overestimate the true variance. That is, there is a small positive bias. I personally would not recommend omitting the f.p.c..

? If N is not large relative to n, then omitting the f.p.c. from the variance formulas can seriously overestimate the true variance. That is, there can be a large positive bias.

? As n N , N - n 0. That is, as the sample size approaches the population size, the N

f.p.c. approaches 0. Thus, in (11) and (12) the variances 0 as n N .

24

2.3.2 SRS With Replacement

? Consider a sampling procedure in which a sampling unit is randomly selected from the population, its y-value recorded, and is then returned to the population. This process of randomly selecting units with replacement after each stage is repeated n times. Thus, a sampling unit may be sampled multiple times. A sample of n units selected by such a procedure is called a simple random sample with replacement.

? The estimators for SRS with replacement are: yU = y

s2 V (yU ) = V (y) = n

? Suppose we have two estimators 1 and 2 of some parameter .

1 is less efficient than 2 for estimating if V (1) > V (2). 1 is more efficient than 2 for estimating if V (1) < V (2).

? For most situations, the estimator for a SRS with replacement is less efficient than the estimator for a SRS without replacement.

? There will be circumstances (such as sampling proportional to size) where we will consider sampling with replacement. Unless otherwise stated, we assume that sampling is done without replacement.

2.4 Two-Sided Confidence Intervals for yU and t

? In an introductory statistics course, you were given confidence interval formulas

y

?

z

s

and

y

?

t

s

(13)

n

n

These formulas are applicable if a sample was to be taken from an infinitely or extremely large population. But when we are dealing with finite populations, we adjust our variance formulas by the finite population correction .

? In the finite population version of the Central Limit Theorem, we assume the estimators

yU = y and t = N y have sampling distributions that are approximately normal. That is,

yU N

N - n S2 yU , N n

and

t N t , N (N - n) S2

n

? For large samples, approximate 100(1 - )% confidence intervals for yU (?) and t ( ) are

For yU :

For t :

(14)

y ? z

N - n s2 Nn

N y ? z N (N - n) s2 n

y ? zs

N -n /n

N

N y ? zs N (N - n)/n (15)

where z is the upper /2 critical value from the standard normal distribution. Or, in standard error (s.e.) notation,

yU ?

t?

For 90%, 95%, and 99%, z = 1.645, 1.96, and 2.576, respectively.

25

? For smaller samples, approximate 100(1 - )% confidence intervals for yU and t are

For yU : y ? t

N - n s2 Nn

For t :

(16)

N y ? t N (N - n) s2 n

y ? ts

N -n /n

N

N y ? ts N (N - n)/n (17)

where t is the upper /2 critical value from the t(n - 1) distribution.

? The quantity being added and subtracted from yU = y or t = N y in the confidence interval is known as the margin of error.

Example: Use the small population data again. For n = 2, t 6.314 for a nominal 90% confidence level.

All Possible Samples and Confidence Intervals from Example Population

Sample y-values

1

0,2

yi yU = y t = N y S2 = s2 S = s V (yU ) V (t)

2

1

5

2

1.4142 0.6

15

90% ci for t (-19.45, 29.45)

2

0,3

3

1.5

7.5

4.5 2.1213 1.35 33.75 (-29.18, 44.18)

3

0,4

4

2

10

8

2.8284 2.4

60 (-38.91, 58.91)

4

0,7

7

3.5

17.5

24.5 4.9497 7.35 183.75 (-68.09, 103.09)

5

2,3

5

2.5

12.5

.5 0.7071 0.15 3.75 (0.27, 24.73)

6

2,4

6

3

15

2

1.4142 0.6

15

(-9.45, 39.45)

7

2,7

9

4.5

22.5

12.5 3.5355 3.75 93.75 (-38.63, 83.63)

8

3,4

7

3.5

17.5

.5 0.7071 0.15 3.75 (5.27, 29.73)

9

3,7

10

5

25

8

2.8284 2.4

60 (-23.91, 73.91)

10

4,7

11 5.5

27.5

4.5 2.1213 1.35 33.75 (-9.18, 64.18)

2.4.1 One-Sided Confidence Intervals for yU and t

? Occasionally, a researcher may want a one-sided confidence interval. There are two types of one-sided confidence intervals: upper and lower.

? Approximate upper and lower 100(1 - )% confidence intervals for yU and t are:

For yU :

For t :

y - ts

N -n /n ,

N

N y - ts N (N - n)/n ,

upper

- , y + ts

N -n /n

N

- , N y + ts N (N - n)/n

lower

where t is the upper critical value from the t(n - 1) distribution.

? If the y-values cannot be negative, replace - with 0 in the lower confidence interval formulas. If the y-values cannot be positive, replace with 0 in the upper confidence interval formulas.

26

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download