Hypothesis Test Notes P-value, Significance Levels ...

Hypothesis Test Notes P-value, Significance Levels & Simulation

Hypothesis Test: Using Random Sample data to decide between two possible views about the population.

Review: Test Statistics A test statistic allows us to measure whether the sample data significantlydisagrees withthe null hypothesis or not.

? Test Statistic falls in the tail determined by critical value Sample data significantly disagrees with null hypothes is.

? Test Statistic does NOT fall inthe tail determined by critical value Sample data does NOT significantly disagree withnull hypothesis.

Big Problem: Sampling Variability!!

Principle of Sampling Variability: Random samples are usually different and random sample statistics are usually very different thanthe populationparameter. Note: Some people refer to Sampling Variability as "random chance".

Deciding Between Two Options Option 1: Is our random sample data different than the populationparameter (null hypothesis) because all random samples are different (samplingvariability)? Inwhichcase the populationparameter and null hypothesis might be correct. OR Option 2: Is our random sample data different than the populationparameter (null hypothesis) because the population parameter and the null hypothesis is wrong.

Dealing with the Two Options Key Question: Could my sample data be different than0 because of samplingvariability? (Could the sample data have occurred by random chance?) Think of samplingvariability (random chance) as a confounding variable. In order to show that the population parameter and the null hypothesis is wrong (option 2), we have to make sure that the reason the sample is different is not sampling variability.

In other words we have to make sure option1 is not correct (or at least highly unlikely), to be able to say that population parameter and the null hypothesis is probably wrong (option 2). In that case, we "Reject the Null Hypothesis". If we cannot rule out option1, we will never know for sure which option is probably correct. The sample data disagrees with the null hypothesis so0 might be wrong. But the sample data might just be different because of samplingvariability indicating 0 might be correct. What a Mess!!!!

P-value to the rescue!!

P-value canhelpus understand sampling variability and decide between the two options.

Definition P-value : The probability of getting the sample data or more extreme because of sampling variability if the null hypothesis is true. Some Stat Books write the definition this way. P-value : The probability of getting the sample data or more extreme by random chance ifthe null hypothesis is true.

Probability & Logic principle: If the probabilityof an event happening is very low, but the event keeps happening, then we should lookfor a different explanation. Our assumption about how that event works might be wrong.

Assumption: Suppose the population parameter in 0 is correct The P-value calculates the probability of getting the sample data because of samplingvariability based on that assumption.

Low P-value (Sampling variability is unlikely.) If the P-value is verylow (close to zero) , then the sample data probably did not happenby sampling variability (random chance). A low P-value rules out sampling variability. Since the sample data probablydidnot occur by sampling variability, the only other option is that the null hypothesis must be wrong. When that happens we say we "Reject the Null Hypothesis". It alsoimplies that the alternative hypothesis is probablycorrect.

High P-value (Could be sampling variability)

If the P-value is high, then the sample data couldhave occurred just because of sampling variability. Since sampling variability might or might not be involved, we will not be able to decide whether the null hypothesisis right or wrong. Which means we also will not be able to decide whether the alternative hypothesis is right or wrong. When this happens, we say we "Fail to Reject the Null Hypothesis". We cannot decide between the null and alternative hypotheses.

Important Notes:

? "Failing to reject 0" does not mean that 0 is true!! It means we cannot tell if the population parameter in 0 is right or wrong. Sampling Variabilityhas struck again.

? A low P-value occurs when the sample value significantly disagrees with the population value in the null hypothesis. In other words a low P-value corresponds with a large test statistic. Both mean that the population parameter in the null hypothesis is probably wrong.

? A high P-value occurs when the sample value is pretty close to the population value in the null hypothesis. In other words a high P-value corresponds with a small test statistic. The population parameter might be correct, but because of sampling variability we cannot tell.

Significance Levels

Sometimes a P-value might be border line. Remember we want the P-value to be low (close to zero) to insure that the sample data did not occur because of sampling variability. But how low do we need it?

Significance Levels (also called "alpha levels")

(Greek Letter Alpha)

Significance levels() are a number we cancompare the P-value too. We will alsosee later they are also associated with avoiding certain types of errors instatistics.

Remember confidence levels? Significance levels () are the opposite of confidence levels (1 - ). If you want to be 95% confident for example the significance level would be 100%-95% = 5%. This is the most common significance

level used.

Common Confidence Levels andSignificance Levels.

Confidence Level (1 - )

Significance Level ()

90% (0.90)

10% (0.10)

95% (0.95)

5% (0.05)

99% (0.99)

1% (0.01)

So before you do your hypothesistest you shouldchoose whichsignificance level youwant to use. If youare unsure, use 5% as this is the most common.

Using Signficance Levels

If the P-value significance level, Reject the null hypothesis. (P-value is low enoughto rule out samplingvariability)

If the P-value > significance level, Fail to reject the null hypothesis. (P-value is too high. Sample data may have occurredbecause of sampling variability.)

P-value Summary

Low P-value (Less thanor equal to the Significance Level)

? Sample data significantly disagrees with the null hypothesis. (Sample data significantly disagrees with the population parameter.)

? Sample data probably did not happen because of samplingvariability. (The sample data probably didnot happen by random chance.)

? Reject 0

High P-value (Higher than the Significance Level)

? Sample data does NOT significantlydisagree with the null hypothesis. (Sample data close to the population parameter.)

? Sample data could have happened because of sampling variability. (The sample data couldhave happened by random chance.)

? Fail to reject 0 (This does NOT mean 0 is correct! It means we don't know.)

Example 1

We used to think that the populationmean average typing speed for all U.S. adults is about 40 (words per minute), but now we think the average typing speed has decreased. We took a large random sample in order to test this claim. Our sample mean was 38 (words per minute). The P-value was 0.216 andthe significance level was 5%.

0 : ? = 40 : ? < 40 (CLAIM)

Convert the P-value intoa percentage.

Compare the P-value to the significance level. Is the P-value large or small?

Does the sample data significantlydisagree withthe null hypothesis?

Could the sample data have occurred because of sampling variability?

Should we reject 0 or fail to reject 0 ? Explainwhy.

Convert the P-value intoa percentage. P-value = 0.216 = 21.6%

Compare the P-value to the significance level. Is the P-value large or small? P-value (21.6%) higher than significance level (5%). This is a highP-value!! (Bad)

Does the sample data significantlydisagree withthe null hypothesis? Sample data does NOT significantlydisagree. They are relatively close. Remember a high P-value means that the test statistic will be smaller than the critical value.

Could the sample data have occurred because of sampling variability? Yes. The P-value is high, meaning if 0 is true, there was a 21.6% probability of getting the sample data or more extreme because of sampling variability.

Should we reject 0 or fail to reject 0 ? Explainwhy. Fail to reject 0. P-value is high and sampling variability might be involved. We will not be able to tell if the null is right or wrong.

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download