Simple Linear Regresion



Partitioning of Sums of Squares in Simple Linear Regression

George H Olson, Ph. D.

Leadership and Educational Studies

Appalachian State University

The parametric model for the regression of Y on X is given by

Yi = α + βXi + εi.. (1)

The model for the regression of Y on X in a sample is

Yi = a + bXi + ei. (2)

Calculation of the constants in the model:

the slope is given by

b = [pic], where [pic], (3)

and the intercept by

[pic]. (4)

A closer look at the regression equation, [pic] leads us to

[pic] (5)

Partitioning the Sum of Squares, [pic]

First, consider the following identity

[pic]. (6)

If we subtract [pic]from each side of the equation, we obtain

[pic] (7)

After squaring and summing, we have

[pic] (8)

or, after simplifying[1],

[pic] (9)

Where SSreg = regression sum of squears, and SSres = residual sum of squares.

Dividing (9) through by the total sum of squares, SStot (= [pic]) gives

[pic] (10)

or

[pic] (11)

A computational example

It is often useful to devise simple computational examples, such as the following:

|Y |X |

|3 |1 |

|1 |0 |

|0 |1 |

|4 |-1 |

|5 |2 |

The means of the two variables, Y and X are

[pic]

Having computed the means, we now compute the deviations, squares of deviations, and cross-products of deviations

|Deviations, squares, and cross-products |

|Y |y |y2 |X |x |x2 |xy |

|3 |.4 |.16 |1 |.4 |.16 |.16 |

|1 |-1.6 |2.56 |0 |-.6 |.36 |.96 |

|0 |-2.6 |6.76 |1 |-1.6 |2.56 |4.16 |

|4 |1.4 |1.96 |-1 |.4 |.16 |.56 |

|5 |2.4 |5.76 |2 |1.4 |1.96 |3.36 |

The sums of squares and cross-products are computed as

[pic]

and the regression coefficients as

[pic]

and

[pic]= 2.6 – 1.769 * .6 = 1.061 = 1.539

The regression equation can now be written as

[pic]

From an earlier equation (9) we obtain

[pic] (12)

Note that we could also have computed

[pic] (13)

An alternative calculation is given by

[pic] (14)

Hence,

[pic] (15)

The equation for the Pearson correlation is

[pic] (16)

Therefore, SSres can also be computed as

[pic] (17)

Appendix

Beginning with,

[pic]

we need to show that

[pic]

Recalling (3, 4 & 5) we can write,

[pic]

-----------------------

[1] See the appendix to see how the equation simplifies.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download