Types of Analyses - University of Kentucky



Chapter 14

Types of Analyses

Item Responses and Univariate Analysis

The first step is to go just a short way beyond the raw data on questionnaires by producing column, row, group, and subgroup tabulations and percentages, often called a "code book." The codebook tells how people answered each item on the questionnaire by frequencies and percentages for each possible response category. Going one step further in the data analysis, evaluators can compute descriptive statistics and other indicators that help describe the frequency distributions.

Bivariate Analysis and Comparison of Two Groups

Comparisons between groups of respondents can be made. If evaluators want to study the relationship between two variables, they use correlational techniques, which show that a change in one variable is associated with a change in another. For example, we might want to determine whether the performance of the Federal Aviation Administration's flight-station service specialists decreases appreciably with age. We would plot the performance scores of specialists of various ages and see whether performance is related to age. We might use an analytic technique such as correlational analysis, which shows the degree to which two variables are related. Or we might compare the differences between two groups rather than the association between variables. For example, we might compare the performance of younger specialists with that of older specialists. Other primary analysis techniques would include cross tabulations, chi-square comparisons, "t" tests, and analyses of variance.

Multivariate Analysis and Comparison of Multiple Groups

This level of analysis is used when what is wanted is look at the associations between more than two variables or at differences between more than two groups. For example, we might want to study the effect of age and experience on Federal Aviation Administration specialists' performance or the effect of age, experience, training and education, and recency of training and education all together. Here, we could use such multivariate techniques as partial correlations, multiple regression analysis, and factor analysis. We could also compare performance by looking at the differences between groups that have varying levels of each trait (older and experienced, younger and experienced, older with limited experience, younger with limited experience, and so on). We might use such techniques as multiple analysis of variances, discriminant analysis, linear structural relations, or log linear analysis.

Choice of Analysis Methods

The choice of data analysis methods depends largely on the evaluation questions and subject matter under study and on the type of variables and what levels of measurement they satisfy. For example, if we had a question about whether the performance of Federal Aviation Administration specialists is different at different ages, and if we had reason to believe that performance was related to age and little else, a simple correlational analysis would reveal the degree of the relationships. But the matters GAO studies are usually more complicated than this, so we would expect other variables such as experience, education, training, and recency of education and training to be related to performance. We would need then to perform multivariate analysis in order to determine the relationships of the variables. Likewise, it might be important to compare performance across several groups rather than to confine the analysis to simple contrasts between pairs. The more complex analyses should usually be undertaken only after the results of simpler analysis have been examined.

Sometimes evaluators have a choice between using associations and using group differences, and sometimes they do not. The shape of the data distribution, the measurement scales, and the plots of the functional relationship between the variables may rule out the use of correlation techniques. For example, sometimes we have to study group differences because the distribution of the observations is not normal; we could not then use certain correlational statistics. Correlational techniques are also inappropriate when the variables are scaled with ordinal data and when the relationships under study are not linear that is, the plot between the variables cannot be transformed into a straight line. It is important to realize that correlational techniques cannot by themselves be used to show causality.

Because questions about cause and effect are sometimes posed, we must note that special designs such as nonequivalent comparison groups, regression discontinuity, and interrupted time-series are usually necessary for establishing causality. The logic of the evaluation design, not the analytic technique, is crucial in drawing inferences about causality.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download