Dimensions of Research



Analysis of Data Using MR and GLMThis lecture: ANOVA via MR and GLMDimensions of Research ProblemsResearch involving only comparison of Group means – Qualitative Factors onlyTraining ProgramPerformanceafter trainingGenderTraining ProgramGenderPerformanceCognitive AbilityCognitive AbilityConscientiousnessGPAEmotional StabilityResearch Involving a mixture of qualitative and quantitative factorsResearch involving only quantitative factorsWe’ll start at the top of this box and work our way through it. We’ll illustrate analysis with MR procedure and with the GLM procedure.Ideally, at the end of this, you’ll be comfortable using either procedure to perform the same analyses.One Qualitative Factor: One Way ANOVAManipulation Check on Faking Data. This example is taken from the Rosetta project data. It is from Raven Worthy’s thesis, completed in 2012. Raven compared three conditions defined by different instructions regarding faking. In each condition, participants were given a personality test (the IPIP 50-item Sample Big 5 questionnaire). Then, they were split into 3 groups.Condition 1: Given instructions to respond honestly. Given a 2nd Big 5 questionnaire (IPIP Other 50 items)Condition 2: Given instructions to fake good. Given same 2nd Big 5 questionnaire.Condition 3: Given instructions to respond honestly but told that highest score would receive a gift certificate.Dependent Variable: The dependent variable is a difference score . . .Mean scores on 2nd questionnaire (all 50 items) minus mean score on 1st questionnaire.Expectation for DV values: 0 means person did not fake on 2nd administration relative to the first.Positive means person faked “good” on 2nd administration relative to first.Negative means person faked “bad” on 2nd administration relative to first.Expectation for Conditions: Condition 1: DV ~~ 0 – Honest responseCondition 2: DV > > 0 – Instructed to fake goodCondition 3: DV > 0 – Incentive to fake good.A snippet of the data (Dependent variable is boldfaced) . . .b5origmean b5othmean b5meandiff ncondit 4.28 4.02 -.26 1 5.20 5.34 .14 1 4.16 3.70 -.46 1 4.68 4.16 -.52 1 5.76 5.62 -.14 1 4.54 4.40 -.14 1 4.96 4.34 -.62 1 5.16 5.06 -.10 1 4.22 3.98 -.24 1 5.36 5.16 -.20 1 5.42 5.38 -.04 1 5.06 4.65 -.41 1 4.82 5.04 .22 1 5.12 5.34 .22 1 4.58 4.46 -.12 1 Preliminary examination of data using dot plots . . .It looks as if the manipulation worked. It appears that the participants in Condition 2 had more positive difference scores (more faking good) than did those in the other two conditions.It also appears that the Incentive condition 3 resulted in more positive scores than Condition 1.But the devil is in the p-values.Note that the ncondit=2 scores seem more variable than the others!!!????Various pieces of information that could/should be presented . . .1. Means and SD’s.2. Plots of means.3. Tests of assumptions.4. Tests of significance.5. Effect sizes and observed power.6. Post hoc tests, if appropriate.The data that were analyzedNote that the dependent variable is a difference score – post instruction minus pre-instruction.The analysis: Analyze -> General Linear Model -> UnivariateWe will always put the name of the nominal / categorical variable defining group membership into the Fixed Factor(s) field.(No group coding variables needed for GLM.)We won’t be using the Random Factor(s) field in this class.The OutputUnivariate Analysis of VarianceBetween-Subjects FactorsNncondit111021083110Descriptive Statistics.This output was obtained by clicking on the [Options] button on the GLM dialog box and then checking Descriptive Statistics.Note this is a quick-and-dirty way to get group means and SD’s.Descriptive StatisticsDependent Variable: b5meandiff nconditMeanStd. DeviationN1-.0861.272051102.6228.844901083.1975.47792110Total.2424.64744328Levene's Test of Equality of Error VariancesaHomogeneity tests.The variances were significantly different across groups. We’ll have to interpret the mean comparisons cautiouslyDependent Variable: b5meandiff Fdf1df2Sig.76.3762325.000Tests the null hypothesis that the error variance of the dependent variable is equal across groups.a. Design: Intercept + nconditThe following is the default output of the GLM procedure.p values. Note that a 2nd p-value is a test that the intercept = 0. Don’t mistake that test for the ones you’re interested in.Test of significance of relationship of DV to all predictors - MR ANOVATests of Between-Subjects EffectsDependent Variable: b5meandiff SourceType III Sum of SquaresdfMean SquareFSig.Partial Eta SquaredNoncent. ParameterObserved PowerbCorrected Model27.727a213.86341.205.000.20282.4091.000Intercept19.643119.64358.382.000.15258.3821.000ncondit27.727213.86341.205.000.20282.4091.000Error109.346325.336Total156.349328Corrected Total137.073327a. R Squared = .202 (Adjusted R Squared = .197)Observed Power: Probability of a significant F if population mean differences were identical to the observed sample mean differences.If we redid the study drawing another sample from the population and the population means were identical to the sample means obtained here, the likelihood of a significant difference would be 1.000.b. Computed using alpha = .05Eta squared. The most common estimate of effect size for analysis of variance. Small: .01Medium: .059Large: .138How did GLM analyze the data?GLM formed two group-coding-variables and performed a regression of b5meandiff onto those variables. The corrected model F is simply the F testing the relationship of RATING to all the predictors – just two in this case. The ncondit F tests the relationship of bemeandiff to the two just the two group-coding variables GLM created to represent differences in ncondit. In this example, those are the only two group-coding variables that were created. In most analyses, there will be many more than just two. What should we conclude?The Oneway ANOVA F is completely valid only when the population variances are all equal. The Levine’s test comparing the variances was significant, indicating that the variances are not equal. This suggests that we should interpret the differences cautiously. We could perform an additional nonparametric test, the Kruskal-Wallis nonparametric ANOVA, for example. Or we could consider transformations of the data to equalize the variances.In this case, however, the effect size is so big that I would feel comfortable arguing that there are significant differences between the means. If the effect size had been small (close to .01) and the p-value large (nearly .05), then I would have less confidence arguing that the means are different..Post-Hoc TestsKnowing that the variances were not equal, I went back to the analysis specification and asked for the Scheffe test, which is a quite conservative post-hoc test and four other post hoc tests designed specifically for situations in which variance are not assumed to be equal. I got the following output . . .Post Hoc TestsThe Dunnett C test gives only lower and upper bounds of a confidence interval for each pair.Nonredundant comparisons p-values are red’d.NconditMultiple ComparisonsDependent Variable: b5meandiff (I) ncondit(J) nconditMean Difference (I-J)Std. ErrorSig.95% Confidence IntervalLower BoundUpper BoundScheffe12-.7090*.07857.000-.9022-.51583-.2836*.07821.002-.4759-.091321.7090*.07857.000.5158.90223.4254*.07857.000.2322.618631.2836*.07821.002.0913.47592-.4254*.07857.000-.6186-.2322Tamhane12-.7090*.08534.000-.9154-.50253-.2836*.05243.000-.4100-.157221.7090*.08534.000.5025.91543.4254*.09320.000.2006.650231.2836*.05243.000.1572.41002-.4254*.09320.000-.6502-.2006Dunnett T312-.7090*.08534.000-.9153-.50263-.2836*.05243.000-.4100-.157221.7090*.08534.000.5026.91533.4254*.09320.000.2007.650131.2836*.05243.000.1572.41002-.4254*.09320.000-.6501-.2007Games-Howell12-.7090*.08534.000-.9113-.50663-.2836*.05243.000-.4075-.159621.7090*.08534.000.5066.91133.4254*.09320.000.2050.645831.2836*.05243.000.1596.40752-.4254*.09320.000-.6458-.2050Dunnett C12-.7090*.08534-.9118-.50623-.2836*.05243-.4082-.159021.7090*.08534.5062.91183.4254*.09320.2039.646931.2836*.05243.1590.40822-.4254*.09320-.6469-.2039Based on observed means. The error term is Mean Square(Error) = .336.*. The mean difference is significant at the .05 level.Homogeneous Subsetsb5meandiffnconditNSubset123Scheffea,b,c1110-.08613110.19752108.6228Sig.1.0001.0001.000Means for groups in homogeneous subsets are displayed. Based on observed means. The error term is Mean Square(Error) = .336.a. Uses Harmonic Mean Sample Size = 109.325.b. The group sizes are unequal. The harmonic mean of the group sizes is used. Type I error levels are not guaranteed.c. Alpha = .05.All of the post-hoc tests agreed that all three means were different from each other.1 vs 2:Significant1 vs 3: Significant2 vs 3: SignificantSo, it appears that the manipulation was successful.When participants were told to respond honestly, they changed their responses the least from pre-instruction to post-instruction (mean diff = -.09).When they were instructed to fake good, they did (mean diff = 0.62).When they were given a mild incentive to fake good, they did, a little (mean diff = 0.20).Wonderful. The manipulation worked!! Profile PlotsNote: SPSS’s plotting algorithm adjusts the scale values so the graph fills as much of the rectangular space as possible.This can make nonsignificant differences look bigger than they really are.It’s important to note that the means that are plotted are estimated means – means estimated assuming all participants had the same value on any covariates. In this case, there were no covariates, so the estimated means are the same as the observed means.But in analyses involving covariates (quantitative predictors), the estimated means and observed means are not equal. GLM’s Group Coding Variables in Oneway ANOVA I said above that GLM forms Group Coding Variables to conduct the ANOVA. What information on them is available? For example, what coding scheme does SPSS use?This example was taken from Aron and Aron, p. 318. Persons in 3 groups rated guilt of a defendant after having read a background sheet on the defendant. For those in Group 1, the background sheet mentioned a criminal record. For those in Group 2, the sheet mentioned a clean record. For those in Group 3, the background sheet gave no information on the defendant's background. Question: Are the differences in guilt ratings significant? Each case is a jury, created just for this experiment.Getting GLM to display Regression rmation on regression analyses performed in GLM including group coding variables is displayed by requesting the “Parameter Estimates” table. It is shown being requested in the dialog box shown on the rightAnalysis Illustrating Default GLM group coding variables.Univariate Analysis of VarianceSince I did not request Eta-squared and Power for this analysis, they are not displayed here.This is new information – the Parameter Estimates Table requested above. The codes are Dummy Codes with the last group as the comparison group. I figured this out by doing a regression analysis with various codes until I found a match.Output from the REGRESSION procedure using the default dummy codesI used the REGRESSION procedure to determine what the GLM procedure’s Parameter Estimates table was giving.Note that the output of the Parameter Estimates box is the same as the regression analysis of dummy codes.So, the default group coding variables estimated in GLM are dummy codes.Analysis Illustrating Requesting your own GLM ContrastsSyntax, if you’re interested.Same data analyzed. This time, I clicked on the [Contrasts...] button and requested Deviation Contrasts. Deviation contrasts are the same as Effects contrasts.UNIANOVA rating BY group /CONTRAST (group)=Deviation /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /PRINT = DESCRIPTIVE PARAMETER /CRITERIA = ALPHA(.05) /DESIGN = group .Univariate Analysis of VarianceSame output as in the above two examples. No effect sizes or power values displayed because I did not ask for them. I did ask for Parameter Estimates.What is displayed in the Parameter Estimates Box is unaffected by the fact that Deviation Contrasts were chosen. What is (always) displayed here is Dummy Codes with the last group as the comparison group.So GLM always displays dummy variable codes in the Parameter estimates box.Analysis Illustrating Requesting your own contrasts continued.Results of user-requested contrasts are presented in the Custom Hypothesis Section.This box is the result of the specification of a contrast other than the “None” default.“Deviation” coding was specified. This is the same as what we have called Effects coding.The Regression Procedure Coefficients box using Effects Coding.Note that the significance values for EC1 and EC2 are identical to the above significance values for Level 1 vs. Mean and Level 2 vs. Mean.Results from REGRESSION, for comparison.The bottom line is that the Parameter Estimates box is pretty useless for ANOVA applications unless you're interested in dummy coding. It is most useful for displaying REGRESSION-like information on quantitative variables.On the other hand, the Contrast Results box gives you a lot of information about your contrasts, although it omits the t-statistic values (???!!!). (But we only look at the p-values anyway, right?)Analysis of Between-Subjects Factorial DesignsIssues Factorial Designs – a reviewDefinitionResearch with 2 or more factors in which data have been gathered at all combinations of levels of all factors.Typical representation is as a Two Way Table.Rows of the table represent one factor – i.e., the Row FactorColumns of the table represent the Column Factor.Cells represent individual groups of persons observed at each combination of factor levels.Factor 1Level 1Level 2Level 1Factor 2Level 2Note - each factor varies completely within each level of the other.All levels of each factor appear at all levels of the other(s).Called a completely crossed design because the variation in each variable completely crosses the other variable.Three Way factorial designs are often represented by separate layers of two way tablesExample: Factor 1 = Type of Training Program, say Lecture vs. ComputerizedFactor 2 = Gender; Factor 3 = Job level – 1st line managers vs. middle managersLectureCompMaleFemaleMiddle ManagersLectureCompMaleFemale1st line ManagersAnalyses of 2x2 Factorial Designs using GLMThe dataThe data are from a study by a surgeon at Erlanger on the effect of helmet use on injuries from ATV accidents. The factors investigated here areHELMET: Whether the driver was wearing a helmet or not, with 2 levels.ROLLVER: Whether the ATV rolled over or not, with 2 levels.The dependent variable is log of the Injury Severity Score (ISS). The larger the ISS value, the more severe the injury. The logarithm was used to make the distribution of the dependent variable more nearly symmetric and less positively skewed.FYI - Here’s a comparison of the distribution of raw ISS vs. log ISS scores.Log10 ISSOriginal ISSSkewness values areStatisticsisslgissNValid500500Missing00Skewness2.263-.624Std. Error of Skewness.109.109So, the ISS values are clearly more positively skewed than are the log ISS values which are actually slightly negatively skewed.We’ll analyze the logs of the ISS values.Expectations:I would expect higher log ISS scores for those not wearing helmets.I would expect higher ISS scores for those who did not roll over assuming no rollover represents collision.That is, they’re in the hospital for a reason. If they didn’t roll over, they must have hit something.I would expect the effect of helmet use to be greater among those who did roll (bigger difference between no helmet and helmet) and less among those who did not (smaller difference between no helmet and helmet) – that is, I would expect an interaction of HELMET and ROLLOVER. I could be wrong. Here’s a small part of the data matrix . . .GLM AnalysisThe syntax, if you’re interested . . .GLM lgiss BY helmet rollover /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /PLOT = PROFILE( helmet*rollover rollover*helmet ) /PRINT = DESCRIPTIVE ETASQ OPOWER HOMOGENEITY /EMMEANS = TABLES(helmet*rollover) /CRITERIA = ALPHA(.05) /DESIGN = helmet rollover helmet*rollover INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET INCLUDEPICTURE "C:\\Users\\Michael\\AppData\\Local\\Temp\\SNAGHTML4358f9.PNG" \* MERGEFORMATINET Univariate Analysis of Variance [DataSet3] G:\MdbT\InClassDatasets\ATVDataForClass050906.savManually produced Two-way TableTests significance of interaction: Difference between effect of Helmet in Rollover group vs. effect of Helmet in No R group: .08 vs. .20..20.08No HelmetHelmetRoll.90.82.89No Ro.98.78.95.93.80.91Only the main effect of helmet usage was significant.There was no main effect of rollover – persons who rolled over did not have significantly different mean log ISS scores.There was no interaction of HELMET and ROLLOVER. The effect of wearing a helmet was the not significantly different among those who rolled over vs among those who crashed into something. Note, however, that the there was a large numeric difference - .08 vs .20 – between the two pairs of means. It was large numerically, but not large enough to be statistically significant. Profile PlotsWhich factor should be the horizontal axis? Both.Both plots give the same information in different ways. One might be more useful than the other.Plot 1: Horizontal axis is defined by helmet use. Different heights of the lines (white ellipses are the marginal means) represent the Rollover effect. You can see that they’re not terribly or consistently parison of left side points with right side points (red ellipses are the marginal means) represent the Helmet effect. Larger mean log ISS scores for no helmet group.Lack of parallelness of lines represent the interaction. They’re crossed, but not so nonparallel as to represent a significant interaction.helmet use * rolloverDependent Variable: lgiss helmet userolloverMeanStd. ErrorThe marginal means are identical to the observed means because there are no covariates or other factors to control for.95% Confidence IntervalLower BoundUpper Bound0 no0 no.984.033.9191.0481 yes.903.024.855.9511 yes0 no.776.074.630.9231 yes.819.056.709.929 Plot 2: Horizontal axis defined by RolloverThis plot is probably easier to understand. Different heights of the lines (red ellipses are the marginal means) represent the helmet effect. They’re quite different in height, reflecting the significant effect.Heights of the left points vs. the right side points (white ellipses are the marginal meas) represent the rollover effect. Average height above “No” is about the same as average height above “Yes”.Lack of parallelness represents the interaction. Lines are not parallel but not so different in slope as to represent an interaction although the difference between helmet use and nonuse is numerically (but not significantly) greater for nonrollover accidents. (The opposite of my expectation. OK – I really expected the Rollover accidents to have a greater effect than the NonRoller accidents. Example of post hoc hypothesizing – Monday morning hypotheses.) Analysis of a 2 x 3 Factorial Design using GLM Myers & Well, p. 127Table 5.1 presents the data for 48 subjects run in a text recall experiment. The scores are percentages of idea units recalled. The data were presented at three different rates - 300, 450, or 600 words per minute. The text was either intact or scrambled. How the data should be entered into SPSSRecallratescram72116311571152116911751168117411492171216321482168216521522163214031493136315031543146314631263165124512RateTextG1G2G3G4G5G6CellscramrateG111G212G313G421G522G623_Analysis of the 2x3 Equal N Meyers/Well data using GLMGET FILE='E:\MdbT\P595\ANOVAviaMR\Meyers_well p.127.sav'.UNIANOVA dv BY row col /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /PLOT = PROFILE( col*row ) /EMMEANS = TABLES(row) /EMMEANS = TABLES(col) /EMMEANS = TABLES(row*col) /PRINT = DESCRIPTIVE ETASQ OPOWER HOMOGENEITY /PLOT = SPREADLEVEL /CRITERIA = ALPHA(.05) /DESIGN = row col row*col .Menu Sequence: Analyze -> GLM -> UnivariateCol 1Col 2Col 3MarginRow 166.2559.8743.3856.50Row 254.3849.7545.8850.00Margin:60.3154.8144.6353.25Manually created Two-way table of means.Univariate Analysis of VarianceWhen the interaction is significant, you must use caution in reporting the main effects – because they’re not “main” anymore – each is conditional on the value of the other factor.The F associated with the "Corrected Model" above is the same as the F in the REGRESSION ANOVA box when all variables are in the equation. It's simply the significance of the relationship of Y to all of the variables coding main effects and interactions.The F in the “Intercept” row is the square of the t for the Intercept in the MR. Don’t interpret it.Observed Power: If the population means were equal to this sample’s means, observed power is the probability you would reject if you took a new sample.Estimated Marginal MeansEstimated marginal mean: An estimate of what the mean would be if all scores were equal on all the other factors.Equal to observed marginal means if 1) cell sizes are equal or 2) if there are no other factors.These are "estimated" means for each row, column, and cell. Since sample sizes are equal, and there are no other factors, in this case the marginal means are equal to the observed means.Spread-versus-Level PlotsUsed to help decide whether variability within each cell increases as mean of a cell increases. If so, a transformation would be recommended.I see no trend here, so no transformation will be made.This is a plot of the "estimated" cell means printed in the above table. Since there are no covariates, they're equal to the observed means displayed in the table at the beginning of the output for this analysis.This plot is the plot recommended for 2-way factorial designs to show in graphical fashion the form of main effects and interactions. It shows that there was an interaction of the row and col effects: the difference between rows (red minus green) changed across columns - with a large difference favoring Row 1 (red) in columns 1 and 2, and a small or perhaps insignificant difference favoring Row 2 (green) in column 3. These differences of differences are the interaction. If the differences were all equal, i.e., there were no differences of differences, the interaction would not be significant.Profile PlotsRateSo, words forming text (Row 1) were recalled at a higher rate than words which were scrambled (Row 2) until the rate got so high that neither was recalled well.Rcmdr Analysis of Myers/Well 2 x 3 Factorial Data – Start 10/2/17Data Import Data From SPSS Dataset . . .Change independent variables to factors in RcmdrBefore performing an ANOVA using Rcmdr, make sure that all factors (qualitative independent variables) are recognized as “factors” by Rcmdr.If you’re not sure, simply do the following. Data Manage Variables in Active Dataset Convert Numeric Variables to Factor . . .If a variable is already a factor, it will either have (Factor) after its name or it will not appear in the list of variables.If you see a variable that you wish to be a factor in an ANOVA in the list, select it. Then click on [OK].If you check [Supply level names], you’ll be asked for a value label for each level. Otherwise, Rcmdr will simply use the values of the variable as the labels.Perfoming the ANOVA using RcmdrStatistics Fit Model Linear Model . . .I double-clicked on the names in the scrolling field to put them into the Model formula line.You can simply type them in, if you wish.R uses A:B to represent the interaction term. The Rcmdr output, after clicking OK . . .> LinearModel.6 <- lm(dv ~ row +col +row:col, data=MyersWell)> summary(LinearModel.6)Call:lm(formula = dv ~ row + col + row:col, data = MyersWell)Residuals: Min 1Q Median 3Q Max -17.375 -4.781 2.188 5.844 11.125 This table is like the Parameters table in SPSS. Each line represents a group coding variable.Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 66.250 2.780 23.832 < 2e-16 ***row[T.2] -11.875 3.931 -3.021 0.00428 ** col[T.2] -6.375 3.931 -1.622 0.11238 col[T.3] -22.875 3.931 -5.819 7.24e-07 ***row[T.2]:col[T.2] 1.750 5.560 0.315 0.75450 row[T.2]:col[T.3] 14.375 5.560 2.586 0.01328 * ---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Residual standard error: 7.863 on 42 degrees of freedomMultiple R-squared: 0.5382,Adjusted R-squared: 0.4833 F-statistic: 9.791 on 5 and 42 DF, p-value: 2.963e-06Argh!! This is not like the GLM “Tests of Between Subjects Effects” output.Getting Rcmdr to print an ANOVA summary table . . .I typed anova(LinearModel.6) and then pressed the [Submit] button.The output of the anova command . . .> anova(LinearModel.6)Analysis of Variance TableResponse: dv Df Sum Sq Mean Sq F value Pr(>F) row 1 507.00 507.00 8.2010 0.006508 ** col 2 2027.38 1013.69 16.3970 5.458e-06 ***row:col 2 492.13 246.06 3.9802 0.026127 * Residuals 42 2596.50 61.82 ---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Bingo!! – Same values as the GLM table. (page 24) Using REGRESSION to perform analyses of Factorial Designs - SkipTo assess each factor, Fs with the following form must be computedR2All Factors – R2All except factor of interest, i.e. R2 for just the factor of interest----------------------------------------------------------------------------------Number of GCV’s for factor of interestFFactor of interest =--------------------------------------------------------------------------------------1 – R2All FactorsProblem – how to get REGRESSION to use this as the denominator---------------------------------------------------N – Number of GCV’s for all factors – 1These analyses can be conducted using only the REGRESSION procedure.To get SPSS REGRESSION to create this F, 0) Request SPSS to print F for R2 change.1) Enter ALL GCV’s, but ignore the output associated with this step.2) Remove the GCVs for the 1st factor being tested, again ignoring the output associated w. this step.3) Re-enter them. The significance of F change assesses significance of the 1st factor.4) Remove the GCVs for the 2nd factor.5) Re-enter them. The significance of F change assesses significance of the 2nd factor.6) Remove the GCVs for the interaction.7) Re-enter them. The significance of F change assesses significance of the interaction.Analysis of the 2x2 Factorial using REGRESSION -skip The 2x2 coding.Factor 2: HelmetFactor 1: RolloverG1G2G3G4Factor 1 (Rollover) compares G1+G2 with G3+G4.Factor 2 (Helmet) compares G1+G3 with G2+G4.Interaction compares G1-G2 with G3-G4, i.e., the contrast is (G1-G2)-(G3-G4).To make the REGRESSION output identical to the GLM output, you must use contrast codes - no dummy variables or effect coding variables.Factor 1Factor 2InteractionGCV1GCV2GCV3G1+.5+.5+.25G2+.5-.5-.25G3-.5+.5-.25G4-.5-.5+.25Creating the Contrast coding variables using syntaxrecode rollover (0=-.5)(1=+.5) into gcv1.recode helmet (0=-.5)(1=+.5) into pute gcv3 = gcv1*gcv2.variable labels gcv1 "GCV representing ROLLOVER " gcv2 "GCV representing whether HELMET was used" gcv3 "GCV representing interaction of ROLLOVER and HELMET".Specifying the regression using syntax The increase in R2 associated with each factor must be tested controlling for all other factors.This means that each factor must be added to the equation containing all the other factors.Usually this means that a sequential procedure described below must be followed.The red’d terms below represent the addition of each comparison to the equation containing all the other comparisons.regression variables = lgiss gcv1 gcv2 gcv3 /descriptives = default corr /statistics = default cha /dep=lgiss Note that I entered gcv3 first. Some analysts will enter the interaction comparison first. If it’s not significant, they’ll leave the gcv for that comparison out of the equation. I chose to leave it in./enter gcv1 gcv2 gcv3 /remove gcv3 /enter gcv3/remove gcv2 /enter gcv2 /remove gcv1 /enter gcv1.Note – each effect – the Row Main Effect, the Col Main Effect, and the Interaction Effect – is represent by a collection of gcvs. In this example, however, each collection has just 1 contrast code.The regression output -skipHELMETRolloverRolloverHELMETIn the following, the red’d lines represent the entry of each factor into the equation containing all the other factors. So the significance of the R2 change is assessed for Model 3, 5, and 7.Variables Entered/RemovedaModelVariables EnteredVariables RemovedMethod1gcv3, gcv1, gcv2b.Enter2.bgcv3cRemove3gcv3b.Enter4.bgcv2cRemove5gcv2b.Enter6.bgcv1cRemove7gcv1b.Entera. Dependent Variable: lgissb. All requested variables entered.c. All requested variables removed.-skipModel SummaryModelRR SquareAdjusted R SquareStd. Error of the EstimateChange StatisticsR Square ChangeF Changedf1df2Sig. F Change1.168a.028.021.34930.0283.6703377.0122.157b.025.019.34951-.0041.4741377.2253.168c.028.021.34930.0041.4741377.2254.167d.028.023.34890.000.1331377.7155.168e.028.021.34930.000.1331377.7156.085f.007.002.35261-.0218.2021377.0047.168g.028.021.34930.0218.2021377.004a. Predictors: (Constant), gcv3, gcv1, gcv2Note that each red p-value is the same as the corresponding p-value in the GLM output above.b. Predictors: (Constant), gcv1, gcv2c. Predictors: (Constant), gcv1, gcv2, gcv3d. Predictors: (Constant), gcv1, gcv3e. Predictors: (Constant), gcv1, gcv3, gcv2f. Predictors: (Constant), gcv3, gcv2g. Predictors: (Constant), gcv3, gcv2, gcv1ANOVAaModelSum of SquaresdfMean SquareFSig.1Regression1.3433.4483.670.012bResidual45.997377.122Total47.3403802Regression1.1632.5824.762.009cResidual46.177378.122Total47.3403803Regression1.3433.4483.670.012dResidual45.997377.122Total47.3403804Regression1.3272.6635.451.005eResidual46.013378.122Total47.3403805Regression1.3433.4483.670.012fResidual45.997377.122Total47.3403806Regression.3432.1711.378.253gResidual46.998378.124Total47.3403807Regression1.3433.4483.670.012hResidual45.997377.122Total47.340380a. Dependent Variable: lgissb. Predictors: (Constant), gcv3, gcv1, gcv2c. Predictors: (Constant), gcv1, gcv2d. Predictors: (Constant), gcv1, gcv2, gcv3e. Predictors: (Constant), gcv1, gcv3f. Predictors: (Constant), gcv1, gcv3, gcv2g. Predictors: (Constant), gcv3, gcv2h. Predictors: (Constant), gcv3, gcv2, gcv1-skipIf we’re only interested in the significance of the effects of the two factors and their interaction, then the coefficients box is of little interest to us.Note that each time all three GCVs were in the equation, the collection of p-values at the right was the same as the collection of p-values obtained with GLM.This is because in this particular example, a regression analysis with all 3 GCVs includes the same tests – row main effect, column main effect, and interaction effect - as the GLM analysis.Since each comparison involved on 1 group-coding variable, this analysis could have been done with one equation: an equation with all three GCVs in it, as illustrated by the circled p-values above..But, the use of only one equation only works for 2 x 2 factorial designs. Any designs with more than 2 levels in a factor would require the sequential projess that I followed above. The example below illustrates the need to perform the analysis of more complex designs sequentially.Analysis of a 2 x 3 Factorial design using REGRESSION -skipFactorial Designs in which one or more factors has more than 2 levels.When a one or more of the main effects has more than 2 levels, the analysis using MR gets a little more complicated. This is because the factor with more than 2 levels must be represented by more than 2 or more group-coding variables. And that means that the interaction will also be represented by more than 1 group-coding variable. The result is that the coefficients box will generally NOT give information on the significance of factors in such an analysis.Example of a 2x3 FactorialThe 2x3 TableFactor 2Factor 1G1G2G3G4G5G6The Data EditorGroupFactor 1Factor 2InteractionF1GCVF2GCV1F2GCV2IntGCV1IntCGV2G1.5.66670.33330G2.5-.3333.5-.1667.25G3.5-.3333-.5-.1667-.25G4-.5.66670-.33330G5-.5-.3333.5.1667-.25G6-.5-.3333-.5.1667.25Main Effect of Factor 1: Average of G1,G2,G3) vs. Average of G4,G5,G6Main Effect of Factor 2- 1st Contrast: Average of G1G4 vs Average of G2,G3,G5,G6 - 2nd Contrast: Average of G2,G5 vs. Average of G3,G6Interaction:1st Contrast: (G1 – Av of G2,G3) vs. (G4 – Av of G5,G6) (G1-G2,G3) - (G4-G5,G6)Is the difference between Col 1 and Col’s 2&3 the same across rows?2nd Contrast: G2 – G3 vs. G5 – G6(G2 – G3) – (G5 – G6)Is the difference between Col 2 and Col3 the same across rows?Coefficients can be easily gotten by multiplying the main effect coefficients.The Myers/Wells data Matrix for Regression analysis with contrast coding -skipUsed for Regression analysisUsed for GLM analysis DV ROW COL ROWGCV COLGCV1 COLGCV2 INTGCV1 INTGCV2 72 1 1 .50 .67 .00 .33 .00 63 1 1 .50 .67 .00 .33 .00 57 1 1 .50 .67 .00 .33 .00 52 1 1 .50 .67 .00 .33 .00 69 1 1 .50 .67 .00 .33 .00 75 1 1 .50 .67 .00 .33 .00 68 1 1 .50 .67 .00 .33 .00 74 1 1 .50 .67 .00 .33 .00 49 1 2 .50 -.33 .50 -.17 .25 71 1 2 .50 -.33 .50 -.17 .25 63 1 2 .50 -.33 .50 -.17 .25 48 1 2 .50 -.33 .50 -.17 .25 68 1 2 .50 -.33 .50 -.17 .25 65 1 2 .50 -.33 .50 -.17 .25 52 1 2 .50 -.33 .50 -.17 .25 63 1 2 .50 -.33 .50 -.17 .25 40 1 3 .50 -.33 -.50 -.17 -.25 49 1 3 .50 -.33 -.50 -.17 -.25 36 1 3 .50 -.33 -.50 -.17 -.25 50 1 3 .50 -.33 -.50 -.17 -.25 54 1 3 .50 -.33 -.50 -.17 -.25 46 1 3 .50 -.33 -.50 -.17 -.25 46 1 3 .50 -.33 -.50 -.17 -.25 26 1 3 .50 -.33 -.50 -.17 -.25 65 2 1 -.50 .67 .00 -.33 .00 45 2 1 -.50 .67 .00 -.33 .00 53 2 1 -.50 .67 .00 -.33 .00 53 2 1 -.50 .67 .00 -.33 .00 51 2 1 -.50 .67 .00 -.33 .00 58 2 1 -.50 .67 .00 -.33 .00 53 2 1 -.50 .67 .00 -.33 .00 57 2 1 -.50 .67 .00 -.33 .00 56 2 2 -.50 -.33 .50 .17 -.25 55 2 2 -.50 -.33 .50 .17 -.25 49 2 2 -.50 -.33 .50 .17 -.25 52 2 2 -.50 -.33 .50 .17 -.25 35 2 2 -.50 -.33 .50 .17 -.25 57 2 2 -.50 -.33 .50 .17 -.25 45 2 2 -.50 -.33 .50 .17 -.25 49 2 2 -.50 -.33 .50 .17 -.25 41 2 3 -.50 -.33 -.50 .17 .25 42 2 3 -.50 -.33 -.50 .17 .25 57 2 3 -.50 -.33 -.50 .17 .25 39 2 3 -.50 -.33 -.50 .17 .25 36 2 3 -.50 -.33 -.50 .17 .25 52 2 3 -.50 -.33 -.50 .17 .25 52 2 3 -.50 -.33 -.50 .17 .25 48 2 3 -.50 -.33 -.50 .17 .25_Number of cases read: 48 Number of cases listed: 48The regression analysis of the Myers/Well data -skipregression variables = dv rowgcv colgcv1 colgcv2 intgcv1 intgcv2 /descriptives = default /statistics = default chaEnters all the variables. /dep=dv /enter rowgcv colgcv1 colgcv2 intgcv1 intgcv2 /remove rowgcv /enter rowgcv /remove colgcv1 colgcv2 /enter colgcv1 colgcv2 /remove intgcv1 intgcv2 /enter intgcv1 intgcv2.RegressionThe "/descriptives = " subcommand above causes these two boxes of output to be printed.-skipOverall ANOVAInteraction effectCol main effectRow main effectNot of any use in this analysis.-skipNot of much use in this analysis.WHEW!!Models 3,5, and 7 are all identical. Each has all variables in the equation. The change in R2 when going from Model 2 to 3, or Model 4 to 5, or Model 6 to 7, tests the significance of one of the effects in the factorial design - either a main effect or the interaction effect. If your only interest is in those effects, then the coefficients box won't be of interest to you. You might be interested in the significance of one or more of the individual GCV's presented here, though.-The MEDRES Dataset – A 2x6 Factorial Design Example Included because it illustrates use of built-in orthogonal polynomialsThis example concerns the issue of quality of surgery residents over the years. There was talk that quality of surgical residents has been decreasing. To address this issue, a doctor at a local hospital conducted a survey of resident programs throughout the nation. The survey requested information on residents’ academic credentials. Of interest here are PART1 scores, scores on the major part of a GRE-like exam taken by all residents, and AOA member qualification, whether the resident was in the top 10% of his/her class.Six years’ worth of data were collected from the survey responses. The distributions of PART1 scores for all six years is given below. Be sure to understand that each year represents a different group of residents.199619971998199920002001year300275250225200175part 1 scoresThe distributions of AOA scores (1=In top 10%, 0=Not) is below.The interest of the investigators was in changes, if any, across years. There was also an interest in any differences that might exist between small programs and large programs.Thus, this is a 6 (Year) x 2 (Progsize) factorial design problem with two dependent variables – PART1 scores and AOA.Specifying the GLM Analysis of PART1 scores from the MEDRES DataThe main dialog box.Tells GLM that PART1 is the dependent variable.YEAR and PROGSIZE are the two independent variables, i.e., factors.These are fixed factors. That is, all of their values are included in the data.The Options dialog box.Have GLM compute estimated marginal means for each factor and for each cell if possible.I checked the “Compare main effects” box to see what GLM output will result.Have GLM display descriptive statistics and effect size and power estimates.The Post Hoc dialog box.Since PROGSIZE is a dichotomy, it makes no sense to ask for post hoc comparisons for it. But YEAR is 6-valued, so we can, although we’ll be focusing on the components of trend requested below.The Contrasts dialog box.I’ve requested that GLM test for linear, quadratic, cubic, etc. trends across the six years. The method will be that of orthogonal polynomials. Thankfully, we don’t have to look up and enter the coefficients – they’re built into the program. All we do is ask for Polynomial contrasts.The GLM OutputFirst, the syntax resulting from all the above pulldowns.UNIANOVA part1 BY year progsize /CONTRAST (year)=Polynomial /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /POSTHOC = year ( BTUKEY ) /PLOT = PROFILE( year*progsize ) /EMMEANS = TABLES(year) COMPARE ADJ(LSD) /EMMEANS = TABLES(progsize) COMPARE ADJ(LSD) /EMMEANS = TABLES(year*progsize) /PRINT = DESCRIPTIVE ETASQ OPOWER /CRITERIA = ALPHA(.05) /DESIGN = year progsize year*progsize .Univariate Analysis of VarianceThe descriptive statistics requested by clicking on Options and checking the Descriptive Statistics box.Examination of means doesn’t suggest a clear trend over years. There is, however, a large numeric difference between small and large programs.What follows is the basic output of GLM – the tests of differences associated with the between-subjects factors.Starting at the top . . .Corrected Model: This is the overall ANOVA in that we’ve seen in REGRESSION output. It’s a test of the significance of the relationship of the DV to ALL the group-coding variables created internally by GLM to represent the two factors and their interaction – all 5+1+5 = 11 of them. The conclusion is that PART1 scores are related to the factors.Intercept.This is a test of the null hypothesis that in the population the intercept of the regression of the DV onto the group-coding variables representing the factors is 0. Year.The Year line tests the significance of the relationship of the DV to the 5 group-coding variables created to represent the YEAR factor. There are significant differences in mean PART1 scores across years. (Use Tests of Contrasts below to see in which direction.)ProgsizeThe Progsize line tests the significance of the relationship of the DV to the 1 group-coding variable created to represent the PROGSIZE factor. There is a significant difference in mean PART1 scores between small and large programs. (Looking at the graphs below shows that larger programs have higher mean.)Year * ProgsizeThis line tests the significance of the relationship of the DV to the 5 product variables created to represent the interaction of YEAR and PROGSIZE. Its nonsignificance means that the change across years is the same for small and large programs.Error.The Error line represents the denominator of the F ratio used to test all the hypotheses. Partial Eta squared column. The entries in this column are estimates of effect size. See 510/511 notes for explanations. It can be reported if a journal wants effect sizes. .020 is small. .071 is medium to large. Noncent ParameterThe noncentrality parameter is a quantity that is used to compute power. It is not reported.Observed Power.The entries in this column give the probability of rejecting the null with a new sample if the population values were equal to those found in this sample.Custom Hypothesis Tests – Results for the orthogonal polynomialsThe Custom Hypothesis Tests box gives the results of contrasts you have specified by clicking on the Contrasts button in the main dialog box.Here the linear trend is significant and the contrast estimate is positive, suggesting that mean PART1 scores increased over the 6-year period.The 5th order contrast is also marginally significant. I would ignore it as a chance result.Estimated Marginal Means 1. yearThe estimated marginal means are computed controlling for the other factor, PROGSIZE.They’re computed assuming that each YEAR has the same mix of small and large programs.So these Estimated Marginal Means will be different than the Obtained means printed in the Descriptives output section.Obligatory aggregation of the above specific comparisons into a single overall test.This output is the result of checking the “Compare Main Effects” box.Mean at each level is compared with the mean at every other level.Don’t request this output unless you know what you’re doing.Estimated Marginal Means 2. progsizeThis output is the result of checking the “Compare Main Effects” box.Mean at each level is compared with the mean at every other level.Don’t request this output unless you know what you’re doing.Post Hoc TestsIn view of the significant linear trend found above, the post hoc tests are kind of meaningless.yearHomogeneous SubsetsThese are identical to the observed cell means.If there had been quantitative covariates, they would not have been identical to the observed means.Skip in Fall 10, Fall 11, Fall 13, Fall 14, Fall 15, Fall 16, Fall17 Fall18Analysis of AOA scores.Because the AOA scores are dichotomies, they should be analyzed using Logistic Regression. The results of the analysis using GLM will be presented here for comparison with logistic regression results when we cover it in a few weeks..UNIANOVA aoa BY year progsize /CONTRAST (year)=Polynomial /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /POSTHOC = year ( BTUKEY ) /PLOT = PROFILE( year*progsize ) /EMMEANS = TABLES(year) COMPARE ADJ(LSD) /EMMEANS = TABLES(progsize) COMPARE ADJ(LSD) /EMMEANS = TABLES(year*progsize) /PRINT = DESCRIPTIVE ETASQ OPOWER /CRITERIA = ALPHA(.05) /DESIGN = year progsize year*progsize .Univariate Analysis of VarianceNote that the YEAR and PROGSIZE factors are both significant, and that the YEAR*PROGSIZE interaction is nearly significant. The “almost” interaction will appear in the plots below.Custom Hypothesis TestsProfile PlotsWe won’t try this one using REGRESSION. Although the procedure is straightforward, it’s quite tedious. Review of main tests conducted in Factorial Designs – skipped in 2017, 2018The data below represent a 2 (row) by 3 (column) factorial design.Is number of words recalled affected by Rate of presentation and Intactness of the words? Participants were shown lists of words. Some were intact words, others were scrambled. So the row factor is “Intactness”?? The data were presented at three different rates - 300, 450, or 600 words per minute. So the column factor is Rate. The scores are percentages of idea units recalled.Effects tested in factorial designsWhen data have been gathered in a 2 way factorial design, the following questions are usually asked. (But remember, the fact that the data were gathered factorially doesn’t mean that you have to analyze them factorially.)1. Is there a main effect of the first factor. Is the DV averaged across levels of the 2nd factor related to changes in levels of the 1st factor. This will involve comparison of 50.0 vs. 56.5 from the above example.2. Is there a main effect of the 2nd factor? Is the DV averaged across levels of the 1st factor related to changes in levels of the 2nd factor. E.g., is the mean of Column 1 significantly different from the mean of column 2? This will involve comparisons of 60.3125, 54.8125, and 44.6250 in the above.3 Is there an interaction of the effects of the 1st and 2nd factors. Does the relationship of the DV to the 1st factor change as levels of the 2nd factor change? Do differences between Row means change from one column to the next? If so, there is an interaction. (Alternatively, do differences between Column means change from one row to the next?) This will involve comparing the difference 66.250-54.375 with 59.875-49.75 with 43.375-45.875. If the differences are consistent, there is no interaction. If the differences are different, then there IS an interaction.Higher order factorial designs.With higher order factorial designs, even more questions involving interactions can be asked . . .For a 3 way factorial design1. We test the main effect of Factor A2. We test the main effect of Factor B3. We test the main effect of Factor C4. We test the interaction of factors A & B. Is the effect of A the same across levels of B, or vice versa?5. We test the interaction of factors A & C. Is the effect of A the same across levels of C or vice versa?6. We test the interaction of factors B & C. Is the effect of B the same across levels of C or vice versa?7. We test the interaction of all factors: ABC Is the interaction of A and B different at different levels of C?Significance Testing – a reviewThe general form of a significance test in MRR2 Factor being tested + Factors controlled for – R2 Factors controlled for ---------------------------------------------------------------------Number of variables representing factor being tested.F = --------------------------------------------------------------------------1 – R2All variables---------------------------------------------------------------------N – All variables – 1The numeratorThe numerator of the numerator:: R2 Factor being tested + Factors controlled for – R2 Factors to be controlled forIt’s the change (i.e., increase) in R2 due to the addition of the variable or variables representing the factor being tested. It’s the increase over and above R2 for another set of variables – those representing the factors being controlled for.So, for example, if I have two factors, A and B, and I want to test the significance of Factor A, controlling for Factor B, the numerator of the F statistics will beR2A + B – R2 BIf I have 3 factors and want to test the significance of A controlling for both B and C, then the numerator will be R2A + B + C – R2 B + CThe significance of a variable or factor is always assessed by determining if it adds to R2 over and above a base. If we’re controlling for other variables, the R2 obtained when they’re in the equation is the base. If we’re not controlling for anything, then 0 is the base.Note that the numerator is an increase in R2, from R2 associated with the base to R2 associated with the base PLUS the variable being tested.The denominatorThe denominator of the F statistic is 1 – R2All variables.This quantity should represent random variability in the dependent variable. It’s the proportion of variance left over when we take out all the predictability associated with the variables we’re studying. The smaller it is, the greater the chance that the F will be significant. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download