Choosing a Significance Test
[Pages:6]Choosing a Significance Test
Student Resource Sheet
Student Resource Sheet
Choosing Your Test
Choosing an appropriate type of significance test is a very important consideration in analyzing data. If an inappropriate test is used, the analysis will not only be meaningless, but also misleading. In order to determine which type of statistical test is most appropriate for your analysis, consider the following three questions:
1. What type of data are you analyzing? 2. How many variables are being measured? 3. How many groups are involved?
Type of Data
Is your data categorical or quantitative? Sometimes the answer to that question is as easy as asking if the data consists of words (categorical) or numbers (quantitative), but the distinction isn't always so clear. For example, a high school student's grade level could be thought of as categorical (freshman, sophomore, junior, senior), or it could be considered quantitative (9, 10, 11, 12). The distinction depends on how you, the researcher, will use the data. If you are interested in the proportion of the sample falling into each category, treat the data as categorical. If, on the other hand, you are interested in calculating the average grade level of the students in your sample or you want to know how grade level is related to GPA, treat the data as quantitative.
Table 1: Determining the type of data
Categorical You will analyze how many subjects fall into each category, or what percent fall into each category. Usually not numeric, but can be numeric
Quantitative You will analyze averages, or you will analyze the correlation between two variables. Always numeric
Examples: Political preference Gender Favorite movie genre
Examples: SAT scores Height Cell phone minutes used per month
Number of Variables
How many variables are being measured? You may have collected data comprised of dozens of variables, but when conducting a significance test, you will not necessarily use all variables in a single test. Focus on the specific question the significance test is answering. For example, if you conducted a survey in which you asked each subject their gender, ethnicity, religious affiliation, political preference, age, height, grade level, GPA, SAT scores, favorite movie genre, and number of cell phone minutes used last month, you have collected data comprised of eleven variables. Before conducting a significance test, you need to focus on a specific question, and then determine how many variables are needed to answer that question. As is often the case in statistics, the answer is not always clear cut.
Student Resource Sheet
Table 2: Determining the number of variables
Question How is gender related to political views?
Is there a correlation between GPA and SAT scores? Do males tend to receive higher SAT scores than females?
Number of Variables Two categorical variables (gender, political preference) Two quantitative variables (GPA, SAT score) Two variables, one categorical (gender) and one quantitative (SAT score), or One quantitative variable (SAT math score) across two groups (male and female)
Number of Groups
Another important consideration when deciding which type of significance test is most appropriate is the number of groups involved. Is the analysis being done on a single group of subjects, or are two or more groups being compared? Comparative studies, those that show how one group differs from another, are often more relevant than single-sample studies. If you intend to compare multiple groups, you may choose to sample multiple sub-populations, or you may choose to sample from one large population, and then break the sample into sub-groups. For example, if you want to conduct a study comparing teenage pregnancy rates among different socio-economic groups, you could sample several students from each socio-economic group separately, or you could take one large sample and then separate those subjects into different socio-economic groups.
Is the Test Valid?
Once you have determined which type of test is appropriate for your analysis, it is necessary to determine whether the test will yield valid results for your particular set of data. Each of the following significance tests involves calculations that are based on certain assumptions and conditions. If the necessary assumptions and conditions are not met, then the results of the test will not be valid, and the test should not be used. This can often be overcome, however, by increasing your sample size.
1. The One-Proportion z-Test
Note: the term "success" refers to the category of interest. For example, if you are conducting a one-proportion z-test to determine if more than half of the sample like horror films, then liking horror films is considered a "success" and not liking horror films is considered a "failure."
2. The Two-Proportion z-Test
Student Resource Sheet
Note: the term "success" refers to the category of interest. For example, if you are conducting a two-proportion z-test to determine if more males than females like horror films, then liking horror films is considered a "success" and not liking horror films is considered a "failure." 3. The Chi-Square Goodness-of-Fit Test
Note: To calculate the expected counts, multiply the sample size by the expected percentage. For example, if you are conducting a chi-square goodness-of-fit test to determine if the distribution of AP scores in your high school differs from the national distribution of AP scores, you would multiply the sample size by the percent of the nation scoring a 5, then by the percent of the nation scoring a 4, etc. This would give the expected counts. 4. The Chi-Square Test for Homogeneity
Note: To calculate the expected counts, first organize the data in a two-way table. For each cell, multiply the row total by the column total, and then divide by the table total. This gives the expected count for each cell. 5. The Chi-Square Test for Independence
Note: To calculate the expected counts, first organize the data in a two-way table. For each cell, multiply the row total by the column total, and then divide by the table total. This gives the expected count for each cell.
6. The One-Sample t-Test
Student Resource Sheet
Note: If the sample is sufficiently large (30 or more), the data need not be approximately normal. If you have a moderate sample size (between 15 and 30), construct a modified box plot and verify that there are no strong outliers. If not, you may proceed with the test. If the sample size is small (less than 15), you must not only verify that there are no outliers, but also construct a histogram and verify that the shape of the distribution is approximately symmetric. Alternatively, you could construct a normal probability plot and verify that the plot is linear.
7. The Paired t-Test
Note: If the sample is sufficiently large (30 or more), the data need not be approximately normal. If you have a moderate sample size (between 15 and 30), construct a modified box plot and verify that there are no strong outliers. If not, you may proceed with the test. If the sample size is small (less than 15), you must not only verify that there are no outliers, but also construct a histogram and verify that the shape of the distribution is approximately symmetric. Alternatively, you could construct a normal probability plot and verify that the plot is linear.
8. The Two-Sample t-Test
Note: If the sample is sufficiently large (30 or more), the data need not be approximately normal. If you have a moderate sample size (between 15 and 30), construct a modified box plot and verify that there are no strong outliers. If not, you may proceed with the test. If the sample size is small (less than 15), you must not only verify that there are no outliers, but also construct a histogram and verify that the shape of the distribution is approximately symmetric. Alternatively, you could construct a normal probability plot and verify that the plot is linear.
9. The One-Way ANOVA (Analysis of Variance) F-test
Student Resource Sheet
Note: To determine whether the groups have similar variances, construct a box plot for each group and compare the interquartile ranges. They need not be exactly the same, but should be similar. To determine whether the data for each group are approximately normal, examine the box plot for each group to verify there are no outliers, and examine the histogram for each group to verify the shape is approximately symmetric. Or, for each group, subtract the group mean from the data (producing residuals) and verify normality with a normal probability plot.
10. The Linear Regression t-test
Note: To determine whether the true relationship between the variables is linear, it is sufficient to verify that the scatter plot of the data is roughly linear. To determine whether the variance is equal for all values of x, graph the least-squares regression line along with the scatter plot and verify that the distances between each point and the line are roughly the same. To determine if the data come from a normal population, it is sufficient to verify that the residuals are approximately normal (this can be accomplished by graphing a histogram or normal probability plot of the residuals).
11. Multiple Regression
Note: To determine whether the true relationship between the variables is linear, it is sufficient to verify that the scatter plots of the data (each x against the y) is roughly linear. To determine whether the variance is equal for all values of x, graph the least-squares regression line along with the scatter plots and verify that the distances between each point and the line are roughly the same. To determine if the data come from a normal population, it is sufficient to verify that the residuals are approximately normal (this can be accomplished by graphing a histogram or normal probability plot of the residuals).
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- using your ti 83 84 calculator for hypothesis testing the
- hypothesis testing power sample size and confidence
- testing differences in means the t test s poupakis
- chapter 6 the t test and basic inference principles
- research rundowns quantitative methods significance
- choosing a significance test
- running a t test in excel
- using your ti nspire calculator for hypothesis testing
- unit 7 hypothesis testing practice problems solutions
- how to run statistical tests in excel
Related searches
- reasons for choosing a university
- choosing a college checklist
- reason for choosing a topic
- choosing a college questionnaire
- choosing a college worksheet
- a b test significance calculator
- reasons for choosing a college
- choosing a career worksheet printable
- choosing a college major worksheet
- what to consider when choosing a college
- choosing a university
- choosing a major test