Linear Regression and Support Vector Regression

Linear Regression and Support Vector Regression

Paul Paisitkriangkrai paulp@cs.adelaide.edu.au The University of Adelaide

24 October 2012

Outlines

? Regression overview ? Linear regression ? Support vector regression ? Machine learning tools available

Regression Overview

CLUSTERING

CLASSIFICATION

REGRESSION (THIS TALK)

+ + ++ ++ +

+

+ ++

+ ++ + +

+

+ +

+++

--

-

--

+

++ + +++ +

K-means

? Decision tree ? Linear Discriminant Analysis ? Neural Networks ? Support Vector Machines ? Boosting

? Linear Regression ? Support Vector Regression

Group data based on their characteristics

Separate data based on their Find a model that can explain

labels

the output given the input

Data processing flowchart (Income

prediction)

Raw

Processed

data

data

Transformed +

data

++

+ +++ +

Pre-processing (noise/outlier removal)

Feature extraction and selection

Regression

Sex Age Hei Income ght

M 20 1.7 25,000

F 30 1.6 55,000

M

1.8 30,000

...

Height and sex seem to be irrelevant.

Mis-entry (should have been 25!!!)

Income

+ ++ ++ ++ +

Age

Linear Regression

? Given data with n dimensional variables and 1 target-variable (real number) {(x1, y1),(x2, y2 ),...,(xm, ym )}

Where x n, y

? The objective: Find a function f that returns the best fit. f : n

? Assume that the relationship between X and y is approximately linear. The model can be represented as (w represents coefficients and b is an intercept)

f (w1,..., wn,b) y w x b

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download