Coursera Machine Learning : W6 Evaluating a learning Alogritm

W6 Evaluation

Notes

Evaluation

• Training set (60%), cross-validation set (20%), and test set (20%).
• If a learning algorithm is suffering from high variance, getting more training data is likely to help.
• Small neural network is prone to underfitting

System Design

• Start from simple model
• Plot learning curves to decide if more data, features, etc. are likely to help
• Error analysis: (1) manually examine errors (2) what potential features could help to classify them.
• Skewed Classes: Positive examples# « Negative examples#
• Precission: #True positive/#Predicted positive
• Recall: #True positives/#Actural positives
• F₁ score 2PR/(P+R)