Statistics 203: Introduction to Regression and Analysis of ...
[Pages:16]Statistics 203: Introduction to Regression and Analysis of Variance
Multiple Linear Regression: Diagnostics
Jonathan Taylor
- p. 1/16
Today
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s Splines + other bases. s Diagnostics
- p. 2/16
Spline models
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s Splines are piecewise polynomials functions, i.e. on an interval between "knots" (ti, ti+1) the spline f (x) is polynomial but the coefficients change within each interval.
s Example: cubic spline with knows at t1 < t2 < ? ? ? < th
3
h
f (x) = 0jxj + i(x - ti)3+
j=0
i=1
where
(x - ti)+ =
x - ti 0
if x - ti 0 otherwise.
s Here is an example.
s Conditioning problem again: B-splines are used to keep the model subspace the same but have the design less ill-conditioned.
s Other bases one might use: Fourier: sin and cos waves;
Wavelet: space/time localized basis for functions.
- p. 3/16
What are the assumptions?
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s What is the full model for a given design matrix X ?
Yi = 0 + 1Xi1 + ? ? ? + pXi,p-1 + i
s Errors N (0, 2I). s What can go wrong?
x Regression function can be wrong ? missing predictors, nonlinear.
x Assumptions about the errors can be wrong. x Outliers & influential observations: both in predictors and
observations.
- p. 4/16
Problems in the regression function
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s True regression function may have higher-order non-linear terms i.e. X12 or even interactions X1 ? X2.
s How to fix? Difficult in general ? we will look at two plots "added variable" plots and "partial residual" plots.
- p. 5/16
Partial residual plot
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s For 1 j p - 1 let
eij = ei + j Xij .
s Can help to determine if variance depends on X j and outliers.
s If there is a non-linear trend, it is evidence that linear is not sufficient.
- p. 6/16
Added-variable plot
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s For 1 j p - 1 let H(j) be the Hat matrix with this predictor deleted. Plot
(I - H(j))Y vs.(I - H(j))Xj.
s Plot should be linear and slope should be j. Why?
Y = X(j)(j) + j Xj + (I - H(j))Y = (I - H(j))X(j)(j) + j (I - H(j))Xj + (I - H(j)) (I - H(j))Y = j (I - H(j))Xj + (I - H(j))
s Also can be helpful for detecting outliers. s If there is a non-linear trend, it is evidence that linear is not
sufficient.
- p. 7/16
Problems with the errors
q Today q Spline models q What are the assumptions? q Problems in the regression
function q Partial residual plot q Added-variable plot q Problems with the errors q Outliers & Influence q Dropping an observation q Different residuals q Crude outlier detection test q Bonferroni correction for
multiple comparisons
q DF F IT S
q Cook's distance
q DF BET AS
s Errors may not be normally distributed. We will look at QQplot for a graphical check. May not effect inference in large samples.
s Variance may not be constant. Transformations can sometimes help correct this. Non-constant variance affects our estimates of SE() which can change t and F statistics substantially!
s Graphical checks of non-constant variance: added variable plots, partial residual plots, fitted vs. residual plots.
s Errors may not be independent. This can seriously affect our estimates of SE().
- p. 8/16
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- glm residuals and diagnostics myweb
- statistics 203 introduction to regression and analysis of
- chapter 8 model diagnostics
- using the f test to compare two models
- lecture 7 linear regression diagnostics
- the examination of residual plots
- math 124 draw residuals plots using the data analysis
- residual analysis and data transformations important
- lecture notes 7 residual analysis and multiple
- residual analysis and outliers
Related searches
- introduction to finance and accounting
- to each and everyone of you
- introduction to leadership and management
- regression and correlation analysis examples
- regression and correlation analysis pdf
- introduction to language and linguistics
- introduction to leadership and governance
- introduction to philosophy and logic
- presentation and analysis of data
- introduction to positive and negative numbers
- introduction to food and beverage service
- introduction to probability and mathematical statistics pdf