Handout Linear Regression



Handout Linear Regression – Spring 2016Consuelo Arbona Ph.D.EPSY 8334 University of Houston Definition: Multiple linear regressions allow one to examine the individual and collective contribution of more than one independent variable (or predictor variable) on a dependent variable (also called criterion variable). For example, regression could be used to examine the contribution of gender, ethnic identity and self esteem to depression scores. Typically, one would say depression scores are regressed onto gender, ethnicity, self esteem and ethnic identity scores. Regression assumes that the relations of the independent and dependent variables are linear (positive or negative).Regression line – best line that describes the relation of one or more dependent variables ( X1 X2 X3) to an independent variable Y: Y = b1 X1 + b2 X2 + b3 X3 + b4 X4 + a (constant). The bs are the beta coefficients, or weight of each variable). The larger the b the stronger the unique association of the predictor variable to the criterion. Characteristics: Both criterion and predictor variables in regression analyses should be continuous. Categorical variables can be included as predictors only if they have two levels (e.g. gender: 1 = F; 2 =M). Categorical variables with more than two levels (e.g. ethnicity: White, Black and Hispanic) need to be dummy-coded. One way to do it is to create two dummy variables that use one group- lets say Whites - as the reference group: Var 1 Hisp where the coding goes Hisp = 1 and W =2 and B = 2 ; Var 2 Black where Black =1 and W=2 and H = 2--- so we code three ethnic groups in terms of two variables Hisp with two values 1= Hisp and 2 for every one else and Black where 1 stands for Blacks and 2 stands for every one else. Both Hisp (coded 1,2)and Black (coded 1,2) are entered as variables to account for three ethnic groups.Predictor variables should be highly related to the criterion variable and the correlation of the predictor variables among themselves should be low (to aovid multicollinearity problems). Three types of linear regression analyses: Simultaneous, Stepwise, and HierarchicalSimultaneous – all predictors are entered at once in the equationStepwise – Computer use an algorithm to decide which predictors variables to enter. For example in forward stepwise regression, the variable with the highest correlation with the dependent variable is entered first, followed by the variable that has the highest correlation with the dependent variable once controlling for the fist variable entered and so on. Stepwise methods (there is also backward stepwise) are used for strictly prediction purposes (not to test conceptual models)and with large sample sizes (about 40 cases per predictor). Results typically are very sample specific and typically do not generalize to other samples. Hierarchical – researcher chooses order that variables are entered in the equation, decided according to (1) causal priority – variables presumed to cause other predictor variables are entered first (e.g. parental SES and offspring’s academic attainment), (2) research relevance/theory- those variables that have been studied before are entered first; variables that theory predict should antecede other variables are entered first (3) main effect variables are always entered before interaction effect variables. Hierarchical regression typically is used to test psychological models. Sample size – Depends on the level of power one wants, the expected value of R2 and the number of predictors. Some authors give the following formula to calculate minimum sample size for linear regression (not considering interaction effects): SS = 50 + 8k (k = # of predictors)Interpretation of Regression Table:R2 Equals the proportion of variance in the criterion variable accounted for or “explained” by the linear combination of the predictor variables. It is considered a measure of effect size. (Comparing size of R2 from different studies is tricky because determining the relative magnitude of the R2 requires taking into consideration range of scores in independent variables, number of independent variables in the analyses and sample size for each study.) Δ R2In hierarchical regression, ΔR2 equals the proportion of variance accounted for by a predictor variable (or a collection of predictor variables) entered in one step over and above the proportion of variance accounted by all the predictor variables that have been entered in the previous steps in the equation (in other words, the incremental value of R2 ). (B)Non-standardized Beta Partial regression coefficients (Column labeled B in the SPSS Regression Table refers to the bs in the equation: Y = b1 X1 + b2 X2 + b3 X3 + c). The Non-standardized Betas indicate how much Y (the value of the criterion variable or DV) will change for a unit of change in the predictor variable (IV) when all the other variables are controlled for. Each B is expressed in the metric of the DV (e.g in one variable scores may range from 1 to 5 and another form 1to 10), therefore the values of the Non- standardized Bs cannot be compared with each other. βStandardized Beta (or Beta coefficients ?) are the Beta values standardized (expressed in terms of standard deviation units from the mean of the criterion or DV), so within one regression analyses they can be compared to each other (However, without a test of significance one cannot determine if the differences observed in ? coefficients in a regression output are statistically significant or not).Table 2 from Pina-Watson et al., (2013) study.Interpretation of Hierarchical Regression Table ? ?R?R? DV – Life SatisfactionStep 1 .02 .02R? indicates that the combination of SES and Generation shares 2% of variance in Life Satisfaction (LS). The R? is not stat. sig. –which means that the combination of variables is not associated to LS. The ?s for SES and Gen are not stat. sig. either. SES .08Generation- .09Step 2The ?R? from step 1 to step 2 = .19 and is statistically significant, which indicates that the addition of Indep. Self C. increased by 19% the amount of variance explained in LS above and beyond the control variables. The ??for Indep. Self C is statistically significant, which means that when controlling for the other two variables, Indep. Self C contributes unique variance to LS.SES .01.19***.21***Generation-.03Indep. Self-C.45***Step 3 .10.31***The ?R? is stat sig. This means that controlling for all other variables in the model (those entered in Step 1 and Step 2) Barriers and CDSE, as a set, contributed additional variance to LS.Examination of the ??coefficients in Step 3 shows that when controlling for all variables the model, Indep. Self C., Barriers and CDSE are stat. sig. which means that each one contributed unique variance to LS, in the expected direction SES-.01Generation-.02Indep. Self-C .26***Barriers -.18***CDSE .29***Step 4 .06**.52***If the study would have examined moderation effects, the interaction terms would have been entered in this step- If the ?R? is statistically significant in the step where interaction terms are entered, then, the interaction effect is statistically significant. But is it strong enough?The ? for the interaction term is examined next. If the ? for the interaction term is stat. sig., it means that the relation of the predictor to the criterion is moderated by the moderator.Therefore, in order to claim an interaction effect both, the ?R? in the step where interaction term is entered and the ? for the interaction term must be statistically significantIn the next page this is explained further based on the Wei et al (2012) article.**p < .01, ***p < .001 Wei et al (2012) Moderation Analyses: Class 4Moderation analyses is used to examine to what extent the strength and direction of the co-relation of a predictor variable (perceived racial discrimination—PRDE ) to a criterion variable (PTSD) is different for people who vary in terms of a third variable –moderator (Ethnic Social Connectedness ESC). This refers to interaction effects or the multiplicative effect of the predictor and the moderator on the outcome of interest. In this case, (PRD X ESC)To test the interaction effect, a new variable – called interaction term- is computed multiplying scores in the predictor and the moderator for each participant – (PRD X ESC, in this case). The interaction term is entered in the last step of the hierarchical regression analyses. If the ?R? for that step is statistically significant (indicating that the interaction term explains an additional amount of variance), is the first indication that there may be an interaction effect.The predictor and moderator variables must be entered in earlier steps so the interaction term (so that the ?R? can be examined).In the table above, the ?R? = .02* from step 3 to 4 is statistically significant therefore, the interaction effect may be statistically significant. In step 4, the ? for one of the two interaction terms is also stat. sig. (PRD X Ethnic SC; ? = -.12*). Therefore, the relations of perceived racial discrimination (PRD) to PTSD is moderated by Ethnic Social Connectedness - But how?? Is the moderation effect consistent with the researcher’s hypotheses?? The nature (strength and direction of correlation) of the interaction is revealed by two separate graphs plotting the relation of PRD to PTSD among those who scored either high \(1 SD>mean) or low ( 1SD<mean) in Ethnic SC. What do the graphs show?? How to plot such graph? Interaction Effect: plotting the interaction effect (Class 5) Plot the interaction for the Wei et al (2012) article using the Excel file provided by the Analyses section of Wei et al (2012) to determine if they standardized the variables before conducting the regression.Determine if the article reports results of a two-way or a three-way interaction.Select the appropriate Excel File in the web site – enter the name of the Predictor and moderator variablesConsult Table 2 in the Wei et al (2012) article to obtain the values for the unstandardized (B) values for the interaction effect (the table does not include the intercept/constant- so do not change that value). Describe what the graph shows regarding the relation of discrimination to PTSD symptoms for participants high versus low in social connectedness with their ethnic group.If you have two standardized variables, you can plot your interaction effect by entering the unstandardized Beta regression coefficients (including intercept/constant) in the following worksheet. If you have control variables in your regression, the values of the dependent variable displayed on the plot will be inaccurate unless you also standardize (or centered) all control variables first (although the pattern, and therefore the interpretation, will be correct). Note that the interaction term should not be standardized after calculation, but should be based on the standardized values of the IV & moderator. 2-way_standardised.xls ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download