Regression Estimation - Least Squares and Maximum Likelihood
[Pages:44]Regression Estimation - Least Squares and Maximum Likelihood
Dr. Frank Wood
Least Squares Max(min)imization
1. Function to minimize w.r.t. 0, 1
n
Q = (Yi - (0 + 1Xi ))2
i =1
2. Minimize this by maximizing -Q 3. Find partials and set both equal to zero
dQ =0
d 0 dQ
=0 d 1
Normal Equations
1. The result of this maximization step are called the normal equations. b0 and b1 are called point estimators of 0 and 1 respectively.
Yi = nb0 + b1 Xi Xi Yi = b0 Xi + b1 Xi2
2. This is a system of two equations and two unknowns. The solution is given by . . .
Solution to Normal Equations
After a lot of algebra one arrives at
b1 =
(Xi - X? )(Yi - Y? ) (Xi - X? )2
b0 = Y? - b1X?
X? =
Xi
n
Y? =
Yi
n
Least Squares Fit
Guess #1
Guess #2
Looking Ahead: Matrix Least Squares
Y1 X1 1
Y2 X2
...
=
...
1
1 0
Yn
Xn 1
Solution to this equation is solution to least squares linear regression (and maximum likelihood under normal error distribution assumption)
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- extending linear regression weighted least squares
- restricted least squares hypothesis testing and
- regression estimation least squares and maximum likelihood
- lecture 4 multivariate regression model in matrix form
- week 5 simple linear regression princeton
- multiple linear regression mlr handouts
- linear least squares
- chapter 2 linear regression models ols assumptions and
- lecture 2 linear regression a model for the mean
- chapter 1 linear regression with 1 predictor
Related searches
- how to graph least squares regression line
- equation of least squares regression line calculator
- slope of the least squares regression line
- least squares regression line example
- the least squares regression line calculator
- least squares regression calculator
- least squares regression equation calculator
- weighted least squares regression excel
- least squares regression method
- least squares regression line calculator
- slope of least squares regression line
- how to find least squares regression equation