How to evaluate a classifier in scikit-learn

In this video, you'll learn how to properly evaluate a classification model using a variety of common tools and metrics, as well as how to adjust the performance of a classifier to best match your business objectives. I'll start by demonstrating the weaknesses of classification accuracy as an evaluation metric. I'll then discuss the confusion matrix, the ROC curve and AUC, and metrics such as sensitivity, specificity, and precision. By the end of the video, you will have a solid foundation for intelligently evaluating your own classification model.

Download the notebook: https://github.com/justmarkham/scikit...

== CONFUSION MATRIX RESOURCES ==
Simple guide to confusion matrix terminology: https://www.dataschool.io/simple-guid...
Intuitive sensitivity and specificity:    • EBM 01: Intuitive Sensitivity & Speci...  
The tradeoff between sensitivity and specificity:    • The tradeoff between sensitivity and ...  
How to calculate "expected value" from a confusion matrix: https://github.com/podopie/DAT18NYC/b...
Classification threshold graphic: https://media.amazonwebservices.com/b...

== ROC/AUC RESOURCES ==
ROC Curves and Area Under the Curve:    • ROC Curves and Area Under the Curve (...  
ROC visualization: http://www.navan.name/roc/
ROC Curves:    • ROC Curves  
An introduction to ROC analysis: http://people.inf.elte.hu/kiss/13dwhd...
Comparing different feature sets: http://research.microsoft.com/pubs/20...
Comparing different classifiers: http://www.cse.ust.hk/nevinZhangGroup...

== OTHER RESOURCES ==
scikit-learn documentation on model evaluation: http://scikit-learn.org/stable/module...
Comparing model evaluation procedures and metrics: https://github.com/justmarkham/DAT8/b...
Counterfactual evaluation of machine learning models:    • Michael Manapat: Counterfactual evalu...  

WANT TO GET BETTER AT MACHINE LEARNING? HERE ARE YOUR NEXT STEPS:

1) WATCH my scikit-learn video series:
   • Machine learning in Python with sciki...  

2) SUBSCRIBE for more videos:
https://www.youtube.com/dataschool?su...

3) JOIN "Data School Insiders" to access bonus content:
  / dataschool  

4) ENROLL in my Machine Learning course:
https://www.dataschool.io/learn/

5) LET'S CONNECT!
- Newsletter: https://www.dataschool.io/subscribe/
- Twitter:   / justmarkham  
- Facebook:   / datascienceschool  
- LinkedIn:   / justmarkham