Performance Measures for Machine Learning
[Pages:32]Performance Measures for Machine Learning
1
Performance Measures
? Accuracy ? Weighted (Cost-Sensitive) Accuracy ? Lift ? Precision/Recall
?F ? Break Even Point ? ROC
? ROC Area
2
Accuracy
? Target: 0/1, -1/+1, True/False, ... ? Prediction = f(inputs) = f(x): 0/1 or Real ? Threshold: f(x) > thresh => 1, else => 0 ? threshold(f(x)): 0/1
?(1- (targeti - threshold( f (xri))))2
accuracy = i=1KN N
? #right / #total ? p("correct"): p(threshold(f(x)) = target)
3
Confusion Matrix
Predicted 1 Predicted 0
a
b
correct
True 0 True 1
c
d
incorrect
threshold
accuracy = (a+d) / (a+b+c+d) 4
True 0 True 1
Prediction Threshold
Predicted 1 Predicted 0
0
b
0
d
? threshold > MAX(f(x)) ? all cases predicted 0 ? (b+d) = total ? accuracy = %False = %0's
True 0 True 1
Predicted 1 Predicted 0
a
0
? threshold < MIN(f(x))
? all cases predicted 1
? (a+c) = total
c
0
? accuracy = %True = %1's
5
optimal threshold 82% 0's in data
18% 1's in data
6
threshold demo
7
Problems with Accuracy
? Assumes equal cost for both kinds of errors
? cost(b-type-error) = cost (c-type-error)
? is 99% accuracy good? ? can be excellent, good, mediocre, poor, terrible ? depends on problem
? is 10% accuracy bad? ? information retrieval
? BaseRate = accuracy of predicting predominant class (on most problems obtaining BaseRate accuracy is easy)
8
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- machine learning generative and discriminative models
- government by algorithm artificial intelligence in
- artificial intelligence in insurance deloitte us
- introduction machine learning
- artificial intelligence and machine learning
- can artificial intelligence power future malware
- the history of artificial intelligence
- artificial intelligence machine learning
- what regulators expect from medical device manufacturers
- performance measures for machine learning
Related searches
- machine learning audiobook
- types of performance measures omb
- matlab machine learning pdf
- probability for machine learning pdf
- machine learning testing
- ai vs machine learning vs deep learning
- machine learning vs deep learning
- machine learning and artificial intelligence
- machine learning vs ai vs deep learning
- difference between machine learning and ai
- machine learning neural networks
- machine learning vs neural network