Coursera Machine Learning : W9 Anomaly Detection

W9 Anomaly Detection

Notes

Density Estimation

• Usage: New machine examination, Fraud Detection
• P(x)=p(x1; miu₁,sigma₁²)p(x1; miu₂,sigma₂²)…
• Anomaly Detection Algorithm
  • Choose features xi that is indicative of anomalous examples
  • Fit parameters miu1…miun; sigma1², sigman².
  • Calculate P(xnew) for the new example, if P(xnew) < epsilon
• Evaluation of Algorithm
  • Example: 10000 Normal, 20 Anomaly
  • Training: 6k Normal
  • Cross-validation: 2k Normal + 10 Anomaly
  • Test: 2k Normal + 10 Anomaly
• Possible Evaluation Metrics
  • True positive/false positive/false negative/true negative
  • Precision/Recall
  • F1-score
• Anomaly Detection with multivariate Gaussian
  • Original model: Computational cheaper, better for manually created features, okay with small m
  • Multivariate Gaussian: Automatic correlation caputure, computational expensive, must m»n (m is 10xn or lager)
• aptitude 资质