Simple Linear Regresion
Partitioning of Sums of Squares in Simple Linear Regression
George H Olson, Ph. D.
Leadership and Educational Studies
Appalachian State University
The parametric model for the regression of Y on X is given by
Yi = α + βXi + εi.. (1)
The model for the regression of Y on X in a sample is
Yi = a + bXi + ei. (2)
Calculation of the constants in the model:
the slope is given by
b = [pic], where [pic], (3)
and the intercept by
[pic]. (4)
A closer look at the regression equation, [pic] leads us to
[pic] (5)
Partitioning the Sum of Squares, [pic]
First, consider the following identity
[pic]. (6)
If we subtract [pic]from each side of the equation, we obtain
[pic] (7)
After squaring and summing, we have
[pic] (8)
or, after simplifying[1],
[pic] (9)
Where SSreg = regression sum of squears, and SSres = residual sum of squares.
Dividing (9) through by the total sum of squares, SStot (= [pic]) gives
[pic] (10)
or
[pic] (11)
A computational example
It is often useful to devise simple computational examples, such as the following:
|Y |X |
|3 |1 |
|1 |0 |
|0 |1 |
|4 |-1 |
|5 |2 |
The means of the two variables, Y and X are
[pic]
Having computed the means, we now compute the deviations, squares of deviations, and cross-products of deviations
|Deviations, squares, and cross-products |
|Y |y |y2 |X |x |x2 |xy |
|3 |.4 |.16 |1 |.4 |.16 |.16 |
|1 |-1.6 |2.56 |0 |-.6 |.36 |.96 |
|0 |-2.6 |6.76 |1 |-1.6 |2.56 |4.16 |
|4 |1.4 |1.96 |-1 |.4 |.16 |.56 |
|5 |2.4 |5.76 |2 |1.4 |1.96 |3.36 |
The sums of squares and cross-products are computed as
[pic]
and the regression coefficients as
[pic]
and
[pic]= 2.6 – 1.769 * .6 = 1.061 = 1.539
The regression equation can now be written as
[pic]
From an earlier equation (9) we obtain
[pic] (12)
Note that we could also have computed
[pic] (13)
An alternative calculation is given by
[pic] (14)
Hence,
[pic] (15)
The equation for the Pearson correlation is
[pic] (16)
Therefore, SSres can also be computed as
[pic] (17)
Appendix
Beginning with,
[pic]
we need to show that
[pic]
Recalling (3, 4 & 5) we can write,
[pic]
-----------------------
[1] See the appendix to see how the equation simplifies.
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- simple linear regression test statistic
- simple linear regression hypothesis testing
- simple linear regression null hypothesis
- simple linear regression model calculator
- simple linear regression uses
- simple linear regression model pdf
- simple linear regression practice problems
- simple linear regression least squares
- simple linear graph generator
- simple linear regression excel
- matlab simple linear regression
- simple linear regression